Qorus Integration Engine®  4.0.3.p2_git
Designing and Implementing Workflows

Table of Contents

Designing Workflows

See also

Overview

Workflows are made up a set of interdependent steps that each perform one traced and restartable action. To design a workflow, a series of logical steps and their dependencies must be defined and then represented by a workflow definition file. Once the high-level design for the workflow has been done, then the logic for the steps can be implemented and the workflow definition and the functions can be loaded into the database and executed.

The following table defines the major elements used when designing and implementing an Qorus workflow.

Qorus Workflow Elements

Element Description
Step The lowest element in a workflow, represents one traced and restartable action. Each step is defined by at least primary step code containing the logic for the step, and optionally other code attributes (such as validation code, run when the step is run in error recovery mode, or asynchronous back-end code, required for asynchronous steps) and other option attributes.
Workflow The workflow is the highest level element defining the logic as a set of steps and inter-step dependencies, along with other attributes; workflows process workflow order data instances that in turn contain the data and the status of processing (status of all steps). A running workflow is called a workflow execution instance and can be run either in batch mode (OMQ::WM_Normal), batch recovery mode (OMQ::WM_Recovery), or synchronous mode.
Queue Asynchronous steps require a queue for linking the associated step data created by the front-end logic with the back-end code.
Workflow Synchronization Events Workflow synchronization event steps allow multiple workflow orders to synchronize their processing based on a single event

Workflow Error Handling and Recovery

Qorus includes a framework for defining error information and raising errors. If a workflow defines an errorfunction in the workflow definition, this function is executed by oload, and the error information is loaded in the system database in the GLOBAL_WORKFLOW_ERRORS and WORKFLOW_ERRORS (see Global and Workflow-Specific Error Definitions for more information). The return value of this function is a hash describing how the system should act when certain errors are raised when processing workflow order data.

Workflows raise errors by calling WorkflowApi::stepError() (in Qore code), WorkflowApi.stepError() (in Java code) or by throwing an appropriate exception.

In the case an exception is thrown, for Qore exceptions, the exception err code is used as the error name, for Java exceptions, the exception full class name (ex: "javax.xml.soap.SOAPException") is used as the error name; the system will then use the error name as the hash key to look up error information in the value returned by the errorfunction.

Note
An error will only affect the step’s status if it has a severity of OMQ::ES_Major or OMQ::ES_Fatal, all other severity codes will cause the error to be logged but the status of the step to be unaffected.

To allow a workflow to recover gracefully from an error, implement validation code for each step. Validation code allows workflows to recover gracefully from errors such as lost request or response messages or temporary communication failures without requiring manual intervention.

Global and Workflow-Specific Error Definitions

By default, error definitions are global. A global definition is a workflow error definition that applies to all workflows. Workflow-specific error definitions apply only to a particular workflow configuration.

There are three ways to create workflow-specific error definitions:

The last point above implies that if two or more workflows define the same error with different attributes (but leave the error's "level" option either unset or assigned to the default: OMQ::ErrLevelAuto), the first error will be a global error, and each time the other workflows define the error with a different set of attributes, those new errors will be workflow-specific error definitions.

Note
Workflow error configurations are considered to be managed by operations, which means that once a workflow has been loaded into Qorus, if its error configuration is updated with the API, then those API-driven changes are persistent and will not be overwritten by subsequent loads of the workflow by oload.

Default Global Workflow Error Definitions

Qorus includes default error definitions for common technical errors that should normally result in a workflow order instance retry and also template errors that can be re-used in workflows as needed.

The following table lists the default errors delivered with Qorus; note that workflows with a severity level of OMQ::ES_Warning ("WARNING") only result in a warning error record but do not affect the logic flow of the workflow.

Default Global Workflow Error Definitions

Error Severity Status Business Description
SOCKET-SSL-ERROR OMQ::ES_Major OMQ::StatRetry False An SSL error occurred while sending or receiving data on a socket
SOCKET-SEND-ERROR OMQ::ES_Major OMQ::StatRetry False An I/O error occurred while sending socket data
SOCKET-RECV-ERROR OMQ::ES_Major OMQ::StatRetry False An I/O error occurred while receiving socket data
SOCKET-TIMEOUT OMQ::ES_Major OMQ::StatRetry False The socket transfer exceeded the timeout period
SOCKET-CONNECT-ERROR OMQ::ES_Major OMQ::StatRetry False An I/O error occurred while attempting to connect to a remote socket
HTTP-CLIENT-RECEIVE-ERROR OMQ::ES_Major OMQ::StatRetry False I/O or protocol error occurred while reading HTTP data
SOCKET-HTTP-ERROR OMQ::ES_Major OMQ::StatRetry False An I/O or protocol error occurred while reading HTTP data on the socket
FTP-RECEIVE-ERROR OMQ::ES_Major OMQ::StatRetry False An I/O or protocol error occurred while reading FTP data
FTP-CONNECT-ERROR OMQ::ES_Major OMQ::StatRetry False An error occurred connecting to the FTP server
FTPS-SECURE-DATA-ERROR OMQ::ES_Major OMQ::StatRetry False An I/O or protocol error occurred on the secure FTP data channel
FTP-DATA-SOCKET-CLOSED OMQ::ES_Major OMQ::StatRetry False The FTP data channel was closed prematurely
FTP-DATA-SOCKET-RECV-ERROR OMQ::ES_Major OMQ::StatRetry False An I/O error occurred on the FTP data channel
FTP-TIMEOUT-ERROR OMQ::ES_Major OMQ::StatRetry False the FTP transfer exceeded the timeout period
SSH2-ERROR OMQ::ES_Major OMQ::StatRetry False An error occurred communicating with the remote server
SFTPCLIENT-TIMEOUT OMQ::ES_Major OMQ::StatRetry False A timeout occurred communicating with the remote server
SFTPCLIENT-LIST-ERROR OMQ::ES_Major OMQ::StatRetry False An error occurred communicating with the remote server in the Qore::SSH2::SFTPClient::list() method
SFTPCLIENT-LISTFULL-ERROR OMQ::ES_Major OMQ::StatRetry False An error occurred communicating with the remote server in the Qore::SSH2::SFTPClient::listFull() method
SFTPCLIENT-CHMOD-ERROR OMQ::ES_Major OMQ::StatRetry False An error occurred communicating with the remote server in the Qore::SSH2::SFTPClient::chmod() method
SFTPCLIENT-MKDIR-ERROR OMQ::ES_Major OMQ::StatRetry False An error occurred communicating with the remote server in the Qore::SSH2::SFTPClient::mkdir() method
SFTPCLIENT-RMDIR-ERROR OMQ::ES_Major OMQ::StatRetry False An error occurred communicating with the remote server in the Qore::SSH2::SFTPClient::rmdir() method
SFTPCLIENT-RENAME-ERROR OMQ::ES_Major OMQ::StatRetry False An error occurred communicating with the remote server in the Qore::SSH2::SFTPClient::rename() method
SFTPCLIENT-REMOVEFILE-ERROR OMQ::ES_Major OMQ::StatRetry False An error occurred communicating with the remote server in the Qore::SSH2::SFTPClient::removeFile() method
SFTPCLIENT-CONNECT-ERROR OMQ::ES_Major OMQ::StatRetry False An error occurred communicating while connecting to the remote server
SFTPCLIENT-GETFILE-ERROR OMQ::ES_Major OMQ::StatRetry False An error occurred communicating with the remote server in the Qore::SSH2::SFTPClient::getFile() method
SFTPCLIENT-GETTEXTFILE-ERROR OMQ::ES_Major OMQ::StatRetry False An error occurred communicating with the remote server in the Qore::SSH2::SFTPClient::getTextFile() method
SFTPCLIENT-PUTFILE-ERROR OMQ::ES_Major OMQ::StatRetry False An error occurred communicating with the remote server in the Qore::SSH2::SFTPClient::putFile() method
SFTPCLIENT-STAT-ERROR OMQ::ES_Major OMQ::StatRetry False An error occurred communicating with the remote server in the Qore::SSH2::SFTPClient::stat() method
SSH2CLIENT-SCPGET-ERROR OMQ::ES_Major OMQ::StatRetry False An error occurred communicating with the remote server in the Qore::SSH2::SSH2Client::scpGet() method
SSH2CLIENT-SCPPUT-ERROR OMQ::ES_Major OMQ::StatRetry False An error occurred communicating with the remote server in the Qore::SSH2::SSH2Client::scpPut() method
SSH2CLIENT-TIMEOUT OMQ::ES_Major OMQ::StatRetry False A timeout error occurred communicating with the remote server
TRANSACTION-LOCK-TIMEOUT OMQ::ES_Major OMQ::StatRetry False a shared database connection could not be acquired within the timeout period
DATASOURCE-RETRY-ERROR OMQ::ES_Major OMQ::StatRetry False An error occurred communicating with the database
RETRY-ERROR OMQ::ES_Major OMQ::StatRetry False (generic) An error occured that results in a workflow order retry; see info for details
REST-CONNECTION-ERROR OMQ::ES_Major OMQ::StatRetry False an error occurred in a REST request
BUSINESS-ERROR OMQ::ES_Major OMQ::StatError True (generic) A business error was detected; see info for details
BUSINESS-RETRY-ERROR OMQ::ES_Major OMQ::StatRetry True (generic) A business error was detected that results in a workflow order retry; see info for details
BUSINESS-WARNING OMQ::ES_Warning OMQ::StatError True (generic) A condition was detected affecting the business logic that results in a warning; see info for details
TECHNICAL-WARNING OMQ::ES_Warning OMQ::StatError False (generic) A technical condition was detected that results in a warning; see info for details
java.io.InterruptedIOException OMQ::ES_Major OMQ::StatRetry False (Java) Signals that an I/O operation has been interrupted. An InterruptedIOException is thrown to indicate that an input or output transfer has been terminated because the thread performing it was interrupted
java.net.SocketTimeoutException OMQ::ES_Major OMQ::StatRetry False (Java) Signals that a timeout has occurred on a socket read or accept
java.net.ConnectException OMQ::ES_Major OMQ::StatRetry False (Java) Signals that an error occurred while attempting to connect a socket to a remote address and port. Typically, the connection was refused remotely (e.g., no process is listening on the remote address/port)
java.net.NoRouteToHostException OMQ::ES_Major OMQ::StatRetry False (Java) Signals that an error occurred while attempting to connect a socket to a remote address and port. Typically, the remote host cannot be reached because of an intervening firewall, or if an intermediate router is down
java.net.HttpRetryException OMQ::ES_Major OMQ::StatRetry False (Java) Thrown to indicate that a HTTP request needs to be retried but cannot be retried automatically, due to streaming mode being enabled
javax.imageio.IIOException OMQ::ES_Major OMQ::StatRetry False (Java) An exception class used for signaling run-time failure of reading and writing operations
java.nio.channels.InterruptedByTimeoutException OMQ::ES_Major OMQ::StatRetry False (Java) Checked exception received by a thread when a timeout elapses before an asynchronous operation completes
java.net.ProtocolException OMQ::ES_Major OMQ::StatRetry False (Java) Thrown to indicate that there is an error in the underlying protocol, such as a TCP error.
java.rmi.ConnectException OMQ::ES_Major OMQ::StatRetry False (Java) A ConnectException is thrown if a connection is refused to the remote host for a remote method call
java.rmi.ConnectIOException OMQ::ES_Major OMQ::StatRetry False (Java) A ConnectIOException is thrown if an IOException occurs while making a connection to the remote host for a remote method call
java.net.SocketException OMQ::ES_Major OMQ::StatRetry False (Java) Thrown to indicate that there is an error creating or accessing a Socket
javax.net.ssl.SSLException OMQ::ES_Major OMQ::StatRetry False (Java) Indicates some kind of error detected by an SSL subsystem
javax.net.ssl.SSLHandshakeException OMQ::ES_Major OMQ::StatRetry False (Java) Indicates that the client and server could not negotiate the desired level of security. The connection is no longer usable
javax.net.ssl.SSLProtocolException OMQ::ES_Major OMQ::StatRetry False (Java) Reports an error in the operation of the SSL protocol
javax.xml.soap.SOAPException OMQ::ES_Major OMQ::StatRetry False (Java) An exception that signals that a SOAP exception has occurred

See the Qorus web UI's "Global Errors" page or the detail page for each workflow to see how error's are configured in Qorus as errors may have been redefined or overridden after the initial installation.

Note
it is recommended that the above errors not be deleted but rather modified, since any new Qorus upgrade will re-write any missing error definitions to the GLOBAL_WORKFLOW_ERRORS table

Workflow Upgrades, Bug Fixes, and Recovery Compatibility

Workflows and steps, along with almost every other object in the Qorus schema, are versioned. When a new version of a workflow is released, either due to a logic upgrade or a bug fix, keep in mind that workflows can only recover data that has been processed by the same version of that workflow.

Note: Keep Recovery Compatibility in Mind when Planning Workflow Updates
Workflows with another workflowid (name and version) cannot recover data from a workflow with another workflowid. Also step changes (new stepid – name and version of the step) will impact workflow recoveries. Keep these facts in mind when planning workflow updates and plan accordingly.

That is; EXAMPLE-WORKFLOW 1.1 cannot recover data processed by EXAMPLE-WORKFLOW 1.0. Furthermore, example-step 1.1 cannot recover data processed by example-step 1.0.

When it is necessary to maintain recovery compatibility in a workflow update, then it will be necessary to redefine steps and workflows with the same version name and number. In these cases, the patch attribute of the object should be updated to reflect the change.

Also keep in mind that new steps added to a workflow will be executed for data that is being recovered even with data that was initially processed by a previous version of the workflow that did not include the new step in its definition.

If a step is removed from a workflow and data must be recovered where that step has a OMQ::StatRetry status, then that data cannot be recovered by a definition of the workflow that no longer includes that step.

These points should be considered carefully when planning workflow updates.

Sometimes it may be necessary to release two or more versions of a workflow when changes must be made; for example one version to make a bug fix in an existing version, and a new version with additional or removed steps also including the bug fix.

Workflows and Order Data

Because a running workflow execution instance can be working on several different orders at once in different threads, accessing workflow data is performed through API calls in order to guarantee that the workflow's program code accesses only the correct data for the current order being processed at all times.

Accessing and processing data is done using the Qorus API as outlined in this section; these APIs set up the data context for each thread so that the correct data is accessed.

Note
Global variables are not allowed in workflow program objects; for a functional equivalent to global variables, see Workflow Execution Instance Data

Workflow Static Order Data

Static data represents the workflow order data being processed. Workflow static order data cannot be updated or deleted by the Qorus workflow API; it is read-only data representing the order data to be processed or fulfilled by the workflow.

APIs:

The information returned by the above function corresponds to the deserialized contents of the field ORDER_INSTANCE.STATICDATA.

Workflow Dynamic Order Data

Dynamic data is associated with the workflow order data instance being processed, but it can be updated and is persistent. Any changes made to dynamic data will be committed to the database before the update method returns, therefore any changes will be available in the future, even in the case of errors and later recovery processing.

Dynamic data is appropriate for storing identifiers and references generated during order processing that are needed in subsequent steps, for example.

Qore API support:

Java API support:

See also
stepdata

Workflow Dynamic Step Data

Dynamic step data, like dynamic order data is associated with the workflow order data instance and also the current step being processed, additionally it can be updated and is persistent like dynamic order data. Any changes made to dynamic step data will be committed to the database before the update method returns, therefore any changes will be available in the future, even in the case of errors and later recovery processing.

Dynamic step data is appropriate for storing information specific to a particular step, particularly user-driven form data with asynchronous steps with the user-interaction flag enabled.

Qore API support:

Java API support:

REST API support:

See also

Workflow Temporary Order Data

Qorus maintains a hash of temporary data associated to the workflow order data instance being processed. This hash can be updated, but it is not persistent, therefore this hash is suitable for temporary data storage only.

This data is lost every time Qorus detaches (i.e. temporarily or permanently stops processing a workflow order data instance, for example, due to an ERROR status and purges the data from the workflow data cache) from a workflow order data instance.

Because temporary data is deleted every time Qorus detaches from a workflow order data instance, it can only be reliably set in the attach logic.

Qore API support:

Java API support:

Workflow Sensitive Order Data

Qorus was designed to allow workflow sensitive order data to be processed while avoiding inadvertent disclosure of this data to unauthorized persons.

Workflow sensitive order data must be processed separately for each data subject stored against the workflow order, this is because each data subject's sensitive data can be queried, updated, or deleted separately across all workflow orders in the system (both in the system schema and any archiving schema).

Workflow order sensitive data is stored separately for each data subject against the workflow order using two identifiers as follows:

  • skey: the sensitive data key type (not treated as sensitive itself, ex: "tax_id", "social_insurance_nr", "ssn", etc)
  • svalue: the sensitive data key value, which is also treated as sensitive itself

The following image provides an overview of a concrete example of sensitive data stored against a workflow order storing sensitive data for at least two data subjects with tax_ids "984.302192.AF" and "739.323.714.BR":

workflow_order_example.png
Qorus Integration Engine Workflow Order Sensitive Data Example

The svalue value, being sensitive itself, should not be stored in static, dynamic data, or step dynamic data. To reference sensitive data from within non-sensitive data, an alias can be used, which is a unique identifier within a workflow order that can be used to uniquely identify sensitive data for a single data subject. Sensitive data aliases are only usable in internal sensitive data APIs.

For example, if a workflow order consists of non-sensitive order information along with a natural person's name and address (which is sensitive) for each order, then the sensitive data alias could simply be the list index (ex: "0", "1", ...).

The following internal workflow APIs provide sensitive data support:

Qore APIs:

Java APIs:

See also

Workflow Execution Instance Data

Workflow execution instance data is stored in a hash maintained by the system. This data is local to the running workflow execution instance pseudo-process and persists until the workflow execution instance terminates.

Any changes made to this data will persist within the running workflow execution instance (pseudo-process) independently of the workflow order data processed.

Because of this, workflow execution instance data is a substitute for global variables in a workflow program. Workflow programs are shared between all running workflow execution instances of that same type (sharing the same name and version and the same workflowid); global variables are not allowed because it is considered unsafe for workflow execution instances to share any common state. If your workflows do need to share some data between execution instances, implement a Qorus service to provide this functionality instead.

Workflow execution instance data will be set in the onetimeinit code (initializing resources for the workflow execution instance), and read by the rest of the workflow.

Qore methods:

Java methods:

Workflow Information

Workflow Metadata

The following properties of the workflow metadata can be returned by the WorkflowApi::getWorkflowMetadata() method (in Qore) or the WorkflowApi.getWorkflowMetadata() method (in Java):

  • "name": name of the workflow
  • "version": version of the workflow
  • "patch": The patch attribute of the workflow
  • "workflowid": ID of the current workflow (metadata ID)
  • "remote": a boolean value giving the remote status of the workflow (if it is running as an independent process or not; see the remote flag)
  • "description": The description of the workflow
  • "cached": The date and time the workflow metadata was read and cached from the database
  • "errorfunction_instanceid": The function instance ID of the error function of the workflow
  • "attach_func_instanceid": The function instance ID of the attach function for the workflow
  • "detach_func_instanceid": The function instance ID of the detach function for the workflow
  • "onetimeinit_func_instanceid": The function instance ID of the onetimeinit function for the workflow
  • "errhandler_func_instanceid": The function instance ID of the error handler function for the workflow
  • "keylist": A list of valid order keys for the workflow
  • "errors": The error hash as returned from the error function
  • "options": A hash of valid workflow options, key = option, value = description

Running Workflow Execution Instance Properties

Properties of the current running workflow execution instance can be retrieved via the WorkflowApi::getWorkflowInstanceData() method (in Qore) or the WorkflowApi.getWorkflowInstanceData() method (in Java); the following keys are present in the response:

  • "dbstatus": the status of the workflow order in the database (for the current status, see the status key, see Workflow, Segment, and Step Status Descriptions for possible values); this will normally be OMQ::StatInProgress, unless called from the Detach Function (first available in Qorus 2.6.2)
  • "external_order_instanceid": The external order instance ID saved against the order, if any
  • "execid": the execution instance ID (workflow pseudo-process ID, first available in Qorus 2.6.0.3)
  • "initstatus": the status of the workflow when it was cached (before it was updated to OMQ::StatInProgress, see Workflow, Segment, and Step Status Descriptions for possible values); note that synchronous workflow orders are created with status OMQ::StatInProgress, so initstatus should always be OMQ::StatInProgress for synchronous orders (first available in Qorus 2.6.2)
  • "instancemode": The mode the workflow execution instance process is running in (OMQ::WM_Normal or OMQ::WM_Recovery) OMQ::StatInProgress, unless called from the Detach Function
  • "mode": The mode the current thread is running in (OMQ::WM_Normal or OMQ::WM_Recovery)
  • "remote": a boolean value giving the remote status of the workflow (if it is running as an independent process or not; see Workflow Remote Parameter)
  • "sync": True if the workflow execution instance is synchronous, False if not
  • "name": name of the workflow
  • "parent_workflow_instanceid": The workflow order data instance ID of the parent workflow if the current workflow is a subworkflow, NOTHING if not
  • "priority": The priority (0 - 999) of the workflow order data instance (first available in Qorus 2.6.0.3)
  • "started": The date/time the workflow order data instance was created (first available in Qorus 2.6.0.3)
  • "status": The current status of the workflow order data instance (see Workflow, Segment, and Step Status Descriptions for possible values); this will normally be OMQ::StatInProgress; first available in Qorus 2.6.2)
  • "version": version of the workflow
  • "workflowid": ID of the current workflow (metadata ID)
  • "workflow_instanceid": The current workflow order data instance ID being processed

Workflow Definition File

Workflow definition files define workflow metadata including the steps and dependencies between steps.

Workflow definition file names must follow the following pattern: *.qwf

Qore-Language Workflow Definition File

In Qore-language workflow definition files, the actual workflow definitions are made by assigning values to the workflows variable in the workflow description file. The top-level keys represent the version(s) of the workflow(s) being described, and the member data of each version key defines that version of the workflow. For a description of this variable, see Workflow Definitions.

The format_version variable in the workflow definition file must be set to "2.6" to declare compatibility with the workflow definition file format documented in this manual.

In order to describe the metadata to the loader (oload), variables with pre-defined names and a pre-defined structure must be defined in this file.

Qore Workflow Definition File Variables

Variable Description
format_version Set to "2.6" to declare compatibility with the oload format defined in this documentation for Qorus 4.0.3.p2_git
workflows Defines workflow metadata
queues Defines asynchronous queues
groups Defines RBAC access groups
events Defines workflow synchronization events

Any other variables, functions, and classes defined in the file will be ignored unless explicitly executed in the script (for example, when building the workflows or queues data structures)

Workflow Definition Example

%new-style
%require-types
%strict-args
%enable-all-warnings
# set format_version to "2.6"
our string format_version = "2.6";
# the groups variable defines RBAC access groups; this group is referenced in the workflow below
our hash<auto> groups.test.desc = "test interfaces";
# the options variable will be used in the workflow definition below
our hash<auto> options = (
"test-mode" : "enable test error generation",
"fast-exec" : "enable simplified execution",
);
# defines workflow EXAMPLE version 1.0
our hash<auto> workflows.EXAMPLE."1.0" = {
# the "desc" key sets the description for the workflow
"desc": "simple example workflow with 3 linear steps",
# the "author" key documents the author of the workflow
"author": "Josephine Programmer",
# the "steps" key defines the steps and describes the step dependencies, in this
# example there are three steps with linear dependencies; the steps are normal
# steps given as classes with the name and version which will describe the name
# and version of the step corresponding to the name and version of the step class
"steps": (
"class:MyTestStep1:1.0",
"class:MyTestStep2:1.0",
"class:MyTestStep3:1.0",
),
# the "workflow-modules" option lists modules providing new step base classes
"workflow-modules": ("MyWorkflowModule1", "MyWorkflowModule2"),
# the "remote" key sets the flag that indicates if the workflow can run in a remote
# "qwf" process if True (the default if not present), or if it must run as an embedded
# workflow in "qorus-core" if False
"remote": True,
# the "autostart" key sets the number of workflow execution instances to be started
# when the system is started; if the system should ensure that this workflow is
# generally running, then set this key to a value > 0
"autostart": 1,
# the "sla_threshold" key sets the amount of time as an integer number of seconds in
# which workflow orders should get a final status and is used for reporting in the
# REST and WebSocket APIs; the default value if not present is 1800 indicating 30
# minutes; the example value here means 20 seconds
"sla_threshold": 20,
# the optional "max_instances" key sets the maximum number of workflow execution
# instances that can be running at one time; note that workflow execution instances
# are capable of processing up to two orders in parallel at any one time (one in
# "normal" mode and one in "recovery" mode).
"max_instances": 1,
# the optional "constants" key sets constant objects to be loaded into the
# workflow's program object; no version may be given for constants
"constants": ("TestConstants1", "TestConstants2"),
# the optional "classes" key defines class objects to be loaded into the
# workflow's program object; no version may be given for classes
"classes": ("TestClass1", "TestClass2"),
# the optional "functions" key defines function objects to be loaded into the
# workflow's program object; no version may be given for functions
"functions": ("TestFunction1", "TestFunction2", "doTest"),
# the optional "mappers" key defines mappers that are registered to the
# workflow which can be retrieved with UserApi::getMapper(); a version key must
# be found in the string;
"mappers": ("my-mapper-1:1.0", "my-mapper-2:1.0",),
# the optional "vmaps" key defines value maps that are registered to the
# workflow which can be used with UserApi::getValueMap(); no version key may be
# given with value maps
"vmaps": ("my-vmap-1", "my-vmap-2",),
# the optional "attach" key defines the attach function for the workflow
"attach": "attach_test:1.0",
# the optional "detach" key defines the detach function for the workflow
"detach": "detach_test:1.0",
# the optional "error_handler" key defines the error handler function
"error_handler": "error_handler_test:1.0",
# the optional "errorfunction" key defines the error definition function
"errorfunction": "test_errors:1.0",
# the optional "onetimeinit" key defines the one-time initialization function
"onetimeinit": "test_onetimeinitialization:1.0",
# the optional "options" key defines workflow options (see options var above)
"options": options,
# the optional "keylist" key defines valid workflow order data keys
"keylist": ("account-id", "customerno"),
# the optional "groups" key adds the workflow to the given RBAC access groups
"groups": "test"
# the optional "statuses" hash defines descriptions for single character custom statuses
"statuses": {
"F": "Filename is already registered with another order. Check it manually",
"I": "Possible inconsistency between stage and cache. Check it manually",
"M": "Missing input file. Order was canceled probably. Check it manually",
"D": "Detach function failed. Check it in the log file",
},
};

See Workflow Parameters for details on how to define workflow in the workflow definition file, and see $OMQ_DIR/examples/user/TEST-WORKFLOWS for some example workflow definition files.

Workflow Definition File Declarations

Queue Definitions

Queues provide the storage and delivery mechanism by which the results of executing an asynchronous event for an asynchronous step are delivered to the right step instance. The queues variable in the workflow definition file must be set to a hash defining queue metadata. The top-level key gives the queue names, and the member data under the queue is a hash with a the desc key, giving a description of the queue.

  • queues."queuename" = ("desc": "description string" ));

If a workflow has at least one asynchronous step, a queue must be defined. A single queue can be used for any number of asynchronous steps, however the keys in a queue must be unique. See the step queue attribute for more information.

The following are examples of queue definitions:

# example Qore queue definition
our hash<auto> queues."my-async-queue".desc = "Asynchronous queue";
See also

RBAC Access Group Definition

In order to allow workflows to declare membership to Interface Groups, the group may be declared in the workflow definition file by assigning a value to the global groups variable.

  • groups."groupname" = ("desc": "description string" ));

The following is an example of a group definition:

# example group definition
our hash<auto> groups."my-group".desc = "my RBAC access group";

Workflow Synchronization Event Type Definitions

Workflow synchronization events are used to synchronize processing from multiple workflow orders; each workflow synchronization event has a key that will be bound to the workflow synchronization event step by calling QorusEventStepBase::bindEvent() or QorusEventStepBase::bindEventUnposted() (in Qore) or QorusEventStepBase.bindEvent() or QorusEventStepBase.bindEventUnposted() (in Java) in the step's primary step code. The workflow synchronization event key is unique within it's workflow synchronization event type; as such, the event type can be compared to a queue name.

The workflow synchronization event type is referenced in workflow synchronization event steps in the eventtype tag.

The actual event type definition is made in the workflow definition file by assigning a value to the global events variable:

  • events."eventname" = ("desc": "description string" ));

The following are examples of workflow synchronization event type definitions:

# example Qore workflow synchronization event type definition
our hash<auto> events."my-event".desc = "my event type";
See also

Custom Statuses

In this hash, descriptions for single character keys of custom statuses available for the workflow can be defined.

# example how to define extended descriptions for some custom statuses
"statuses": {
"F": "Filename is already registered with another order. Check it manually",
"I": "Possible inconsistency between stage and cache. Check it manually",
"M": "Missing input file. Order was canceled probably. Check it manually",
"D": "Detach function failed. Check it in the log file",
)

See

Workflow Parameters

The workflow hash description consists of the step dependencies and other attributes.

The following diagram illustrates a subset of the attributes of a workflow.

workflow-diagram.png
Workflow Metadata Diagram

The "workflows" variable in a Qore workflow definition file must declare a hash that will define workflow metadata.

Note
In a Qore workflow definition file, the top-level keys in the "workflows" global variable represent workflow names; these keys must also be assigned to a hash, where each key is a version of the workflow being defined. The value assigned to each version key must be a workflow description hash, representing the workflow metadata definition.

The workflow description hash has the following keys (optional keys are enclosed in square brackets []):

Here step and function designators (except for in the "functions" tag above) define a specific version of the appropriate object. This is done by specifying a string in the following format: "name:version" or by specifying a hash with name and version keys set to the appropriate values.

Workflow Parameter Descriptions

Parameter Description
author The author of the workflow
steps This is where steps and step dependencies are defined
desc A string giving the description for the workflow
workflow-modules lists modules providing new step base classes for class-based steps
remote flag that indicates if a workflow can run in a remote qwf process or not (see qorus-client.remote for how the default value is assigned by oload)
autostart the number of workflow execution instances to start when the system starts
[sla_threshold] the amount of time as an integer in seconds in which each workflow order should get a final status; if not present the default value of 30 minutes is assumed (see DefaultWorkflowSlaThreshold)
max_instances the maximum number of workflow execution instances that can be started
classes If defined, this list of strings designates the Qore classes that will be loaded into the workflow's program object from the CLASSES table when the workflow is cached (represented by the "Shared Classes" object in the diagram above). Note that only the Qorus class object name should be given as it appears in the database without any version number, the latest version of the class will be loaded into the workflow's Program object.
constants If defined, this list designates the Qore constant definitions that will be loaded into the workflow's program object from the CONSTANTS table when the workflow is cached (represented by the "Shared Constants" object in the diagram above). Note that only the Qorus constant object name should be given as it appears in the database without any version number, the latest version of the constant will be loaded into the workflow's Program object.
functions If defined, this list designates the Qore function definitions that will be loaded into the workflow's program object from the FUNCTIONS table when the workflow is cached (represented by the "Shared Functions" object in the diagram above). Note that only the Qorus function object name should be given as it appears in the database without any version number, the latest version of the function will be loaded into the workflow's Program object
mappers If defined, this list designates the mappers that will be registered with the workflow so they are available with the UserApi::getMapper() call in the workflow's code; the mapper name must be given with an explicit version number in each string (ex: "my-mapper-1:1.0")
vmaps If defined, this list designates the value maps to register with the workflow so they can be used with the UserApi::getValueMap() call in the workflow's code; no version number can be given in the value map names
attach If defined, this string identifies a function that will be executed every time Qorus caches and starts working on ("attaches to") a particular workflow order data instance (represented by the "Attach Logic" object in the diagram above). The function's version may be given by appending a colon and the version identifier to the name as follows: "detach_function_name:1.0"
detach If defined, this string identifies a function that will be executed every time Qorus stops working on ("detaches from") a particular workflow order data instance (represented by the "Detach Logic" object in the diagram above). The function's version may be given by appending a colon and the version identifier to the name as follows: "detach_function_name:1.0".
onetimeinit If defined, this string identifies a function that will be executed once when each workflow execution instance is started (represented by the "Init Logic" object in the diagram above). The function's version may be given by appending a colon and the version identifier to the name as follows: "onetimeinit_function_name:1.0"
error_handler If defined, this string identifies a function that will be executed every time an error is raised by the workflow (represented by the "Error Handler" object in the diagram above). The function's version may be given by appending a colon and the version identifier to the name as follows: "error_handler_function_name:1.0"
errorfunction This string identifies a function that is called by Qorus when the workflow definition is loaded by oload (represented by the "Error Definitions" object in the diagram above). It should return a hash of error information. See Workflow Error Handling and Recovery for more information. The function's version may be given by appending a colon and the version identifier to the name as follows: "error_function_name:1.0"
options The options key should be assigned to a hash defining valid workflow options
keylist This key should be assigned to a list giving valid workflow keys for fast lookups of workflow order instance data
groups list of workflow and service groups the workflow will belong to
statuses a hash defining extended custom statuses descriptions.
Note
It's important to decide carefully whether to use generic functions (or classes) to extend the functionality of a workflow, or to develop a user service. Normally functionality belongs in a user service if it meets one or more of the following criteria:
  1. It will be called from external applications and should be exported through the network API
  2. It will change often or is used by multiple workflows, and it would be desirable to perform live upgrades of the functionality independently of the workflow(s) that depend on it
  3. The functionality is critical and needs to be developed/tested independently of the workflow(s) that depend on it

Workflow Definition Options Key

If the options key is defined in the workflow definition, the workflow will advertise and accept only the options defined under this key as valid options for the workflow (aside from system options that can be overridden at the workflow level, which are always accepted as workflow options). Valid options are given as a hash assigned to this key, where the hash keys are the option names, and the values assigned to the keys are the descriptions of the options.

Here is an example of am options declaration in a Qore workflow definition file:

# example workflow definition with options
our hash<auto> workflows."EXAMPLE-WORKFLOW"."1.0" = (
"desc": "simple example workflow",
"steps": ( ... ),
"options": {
"account": "override default account code",
"summary": "set to True for a summary report",
},
);

Options set on workflows are persistent; the option values are written to the WORKFLOW_OPTIONS table.

Workflow Definition Keylist Key

The workflow definition option keylist defines one or more "order keys" that can be used to quickly look up workflow order data instances. Any number of workflow keys may be given in the workflow definition, in contrast to the single external_order_instanceid that can be given at the workflow order data instance object level and can be used for the same purpose (quickly looking up workflow order data instances from a external key).

Valid keys are defined as per the following examples:

# example Qore workflow definition with a keylist
our hash<auto> workflows."EXAMPLE-WORKFLOW"."1.0" = (
"desc": "simple example workflow",
"steps": ( ... ),
"keylist": ("account_no", "customer_id"),
);

Keys can be set and retrieved by the WorkflowApi::setOrderKeys() and WorkflowApi::getOrderKeys() methods (in Qore) or the WorkflowApi.setOrderKeys() and WorkflowApi.getOrderKeys() methods (in Java), respectively. Key metadata is saved in the WORKFLOW_KEYS table, and workflow order data instance keys are saved in the ORDER_INSTANCE_KEYS table. Note that a single key value can be saved for more than one workflow order data instance; the indexes on the ORDER_INSTANCE_KEYS only enforce that values may not be repeated for the same workflow_instanceid and key name.

The REST API provides methods to lookup workflow orders by workflow keys; see the following API for more information:

Workflow Definition Groups Key

The workflow definition option groups defines one or more Interface Groups that the workflow is a member of. RBAC access groups can also be declared in the workflow file using the global groups variable (see RBAC Access Group Definition).

RBAC access group membership is declared as per the following examples:

# example Qore workflow definition declaring membership in an RBAC access group
our hash<auto> workflows."EXAMPLE-WORKFLOW"."1.0" = (
"desc": "simple example workflow",
"steps": ( ... ),
"groups": "my-group",
);

Workflow Steps

The inter-step dependencies are defined in the steps key in the workflow definition. This section describes how to define step dependencies and simple step definitions. For details on how to define complex steps, see Step Definitions.

The basic format of this data structure is a list. At the top level, a list indicates sequential dependencies, as in the following examples:

# simple Qore example workflow definition
our hash<auto> workflows."EXAMPLE-WORKFLOW"."1.0" = (
"desc": "simple example workflow",
"steps": ("class:MyStep1:1.0", "class:MyStep2:1.0", "class:MyStep3:1.0"),
);

In this example; the workflow EXAMPLE-WORKFLOW 1.0 is created. The step dependencies are linear: MyStep1 has no dependencies and therefore is the starting step, MyStep2 is dependent only on MyStep1, and MyTestStep3 is the final step in the workflow and is dependent only on MyStep2.

simple-steps.png
Simple Workflow Execution Dependency Diagram

In this simple example, 3 steps will be created (or updated if the steps already exist) with the same names and version specifiers as the class names above and placed in simple linear dependency in the steps list.

Note
Each of the step definitions in this simple example is made in the simplest format possible, which is "class:stepname:version". In this case, the step will have the same name and version as the class that defines the step. That means that classes "MyStep1" version "1.0", "MyStep2" version "1.0", and "MyStep3" version "1.0" must also be defined; See array steps, subworkflow steps, asynchronous steps, validation code, function definition file.

It is possible to create more complex step dependencies. For example, if one of the entries in the top-level steps list is itself a list, then all entries in the sublist have the same dependency and therefore will be executed in parallel threads. Furthermore, of one of the entries in the sublist is also a list, then they will be assigned sequential dependencies (i.e.: be executed sequentially) within this sublist. Using this simple syntax, it is possible to define a complex multithreaded workflow with minimum effort.

Here are more complex examples:

# more complex example Qore workflow definition
our hash<auto> workflows."COMPLEX-EXAMPLE-WORKFLOW"."1.0" = (
"desc": "complex example workflow",
"steps": (
{"name": "step1", "classname": "MyStep1:1.0"},
(
(
{"name": "step2", "classname": "MyStep2:1.0"},
{"name": "step3", "classname": "MyStep3:1.0"},
),
{"name": "step4", "classname": "MyStep4:1.0"},
),
{"name": "step5", "classname": "MyStep5:1.0"},
),
);

In this example, MyStep1 will be executed first, then 2 threads will start in parallel. In one thread, MyStep2 and MyStep3 will be executed in sequence. In the other thread, MyStep4 will be executed. Only when all of these steps reach a OMQ::StatComplete status, will MyStep5 be executed as the final step in the workflow.

The execution diagram will look like the following:

complex-steps.png
Complex Workflow Execution Diagram

Using this simple rule of alternating list levels for sequential and parallel execution, it is possible to create very complex multithreaded workflows with minimum effort.


Workflow Modules Parameter

The "workflow-modules" option lists modules providing new base classes for class-based steps as in the following example.

# defines workflow EXAMPLE version 1.0
our hash<auto> workflows.EXAMPLE."1.0" = {
# ...
"workflow-modules": ("MyWorkflowModule1", "MyWorkflowModule2"),
# ...
};

Modules declared like this will be loaded into each workflow's Program object, and their classes can be used as base classes for class-based steps.

See also
Workflow Extension Modules for more information.

Workflow Author Parameter

The "author" value indicates the author of the workflow and will be returned with the workflow metadata in the REST API and also is displayed in the system UI.


Workflow Remote Parameter

The "remote" flag indicates if the workflow will run as an independent qwf process communicating with other elements of Qorus Integration Engine with a distributed queue protocol rather than internally in the qorus-core process.

When workflows run in separate qwf processes, it provides a higher level of stability and control to the integration platform as a whole, as a workflow with implementation problems cannot cause the integration platform to fail.

There is a performance cost to running in separate qwf processes; workflow startup and shutdown is slightly slower, and communication with qorus-core also suffers a performance hit as all communication must be serialized and transmitted over the network.

Furthermore memory usage is significantly higher for interfaces running in separate programs, as all the common infrastructure for each interface must be duplicated in each process.

The default for this option depends on the client option qorus-client.remote (if this client option is not set, then the default value is True).

The remote value can be changed at runtime by using the following REST API: PUT /api/latest/workflows/{id_or_name}?action=setRemote

Note
The remote flag is considered to be managed by operations, which means that once an interface has been loaded into Qorus, if its remote flag is updated with the API, then those API-driven changes are persistent and will not be overwritten by subsequent loads of the interface by oload.

Workflow Autostart Parameter

The workflow "autostart" parameter sets the number of workflow execution instances to be started when the system is started; if the system should ensure that this workflow is generally running, then set this key to a value greater than zero.

If no value is provided for this option, the system will not start the workflow automatically; any workflow execution instances for this workflow must be started manually.

If a non-zero value is provided for this workflow, then the system will attempt to start the workflow at all times if all its dependencies are met, and it is not disabled. Additionally, if the workflow cannot be started for any reason (for example, due to an error in the onetimeinit logic or a dependency error), an ongoing system alert will be raised, which is only cleared when the workflow is successfully started (or the autostart parameter is set to zero).

Note
The workflow "autostart" value is considered to be managed by operations, which means that once a workflow has been loaded into Qorus, if its "autostart" value is updated with the API, then those API-driven changes are persistent and will not be overwritten by subsequent loads of the workflow by oload.

Workflow SLA Threshold Parameter

The workflow "sla_threshold" parameter sets the amount of time as an integer in seconds in which each workflow order should get a final status, where a final status is defined as either COMPLETE or CANCELED; if not present the default value of 30 minutes is assumed (see DefaultWorkflowSlaThreshold).

This value is used to do SLA reporting for workflow orders in the REST and WebSocket APIs.

Note
The workflow "sla_threshold" value is considered to be managed by operations, which means that once a workflow has been loaded into Qorus, if its "sla_threshold" value is updated with the API, then those API-driven changes are persistent and will not be overwritten by subsequent loads of the workflow by oload.

Workflow Max Instances Parameter

The optional "max_instances" key sets the maximum number of workflow execution instances that can be running at one time; note that workflow execution instances are capable of processing up to two orders in parallel at any one time: one in "normal" mode and one in recovery mode.

To serialize workflow processing and ensure that only one order is processed at one time at any particular point in the workflow's logic, set the "max_instances" value to one and use the synchronized keyword on a function or method called in both normal and recovery mode or make a service call to a service method that uses threading primitives to ensure atomicity of operation (or has the service method write lock flag set).

Workflow Configuration Items

Workflows cannot declare configuration items because workflows are just set of steps. Instead it's allowed to set the value of a step configuration item on workflow level at runtime using the operational web UI or the REST API.

Note
All configuration item values are considered to be managed by operations by design, which means that once a configuration item has been loaded into Qorus, any changes to its value with the API are persistent and will not be overwritten by subsequent loads of the configuration item by oload.

OneTimeInit Function

Synopsis
This function is executed once when the workflow execution instance starts, and once again after every workflow instance cache reset.
Note: Acquire Workflow Execution Instance Resources in the OneTimeInit Function
Persistent objects with a high acquisition cost should be acquired in the onetimeinit function. Use the WorkflowApi::updateInstanceData() method (in Qore) or the WorkflowApi.updateInstanceData() method (in Java) to save resources acquired, and WorkflowApi::getInstanceData() (in Qore) or the WorkflowApi.getInstanceData() method (in Java) to retrieve the resources during the workflow's execution.
Function Template
nothing sub functionname() {}
Expected Return Value
NOTHING (any return value will be ignored)
One Time Initialization Function Definition Example
# type: GENERIC
# version: 1.0
# desc: example workflow initialization
%new-style
%require-types
%strict-args
%enable-all-warnings
sub example_workflow_init() {
object obj = acquire_an_expensive_object();
# save it in instance data
WorkflowApi::updateInstanceData({"obj": obj});
}
# END

Attach Function

Synopsis
This function is called when Qorus starts working on a workflow order data instance (when Qorus "attaches" to the workflow order data instance, reading it into memory from the database).

If any error is raised in the attach function (by calling WorkflowApi::stepError() (in Qore code), WorkflowApi.stepError() (in Java code) or by throwing an appropriate exception)), the workflow order data instance will receive an OMQ::StatError status and the attach operation will fail.
Note: Reliably Set TempData in the Attach Function
TempData can only be reliably set in the attach function, because a workflow can be restarted from any step after an error. Therefore you cannot ensure that TempData set in a normal step function will be available when executing the following step, in case an error occurs.
Function Template
nothing sub functionname() {}
Expected Return Value
NOTHING (any return value will be ignored)
Attach Function Definition Example
# type: GENERIC
# version: 1.0
# desc: example attach function
%new-style
%require-types
%strict-args
%enable-all-warnings
sub example_attach() {
log(LL_INFO, "attach function");
}
# END

Detach Function

Synopsis
This function is called when the workflow order data instance status is committed to the database (when Qorus "detaches" from the workflow order data instance). The first argument passed to the function is the workflow order data instance's status (status descriptions) that will be written to the database. The second argument passed is the external order instance ID, if any exists.
Note: Update Processing Status Externally with the Detach Function
Because the detach function is called any time the workflow order data instance status is saved to the database, it can be used to update the status of a workflow order data instance in an external system, if required.
Function Template
nothing sub functionname(string status, *string external_order_instanceid) {}

Detach Function Parameters

Type and Name Description
string status The status being set for the workflow order data instance, see Workflow, Segment, and Step Status Descriptions
__7_ string external_order_instanceid The external key for the workflow data being processed, if any exists, otherwise NOTHING
Expected Return Value
NOTHING (any return value will be ignored)
Detach Function Definition Example
# type: GENERIC
# version: 1.0
# desc: example detach function
%new-style
%require-types
%strict-args
%enable-all-warnings
sub example_detach(string stat) {
log(LL_INFO, "detach function: stat=%n", stat);
}
# END

Error Handler

Synopsis
This function is called when errors are raised. It allows the workflow to do external logging or to take custom actions when errors are raised.
Function Template
nothing sub functionname(string errcode, *hash errinfo, any opt) {}

Error Handler Function Parameters

Type and Name Description
string errcode The error code string (ex: "QORE-EXCEPTION")
__7_ hash errinfo This parameter will only be present if the error has been defined by the Workflow Error Function; if so, the hash should have at least the following keys (key names in square brackets (i.e. [name]) are optional):
- desc: description of the error
- severity: Error Severity Codes
- status: either OMQ::StatRetry (meaning that the step should be retried) or OMQ::StatError
- [retry-delay]: the delay in seconds before the error should be retried (only valid when status is OMQ::StatRetry)
- [business]: if True, this error represents a business error (rather than a technical error); usually implying that there is a problem with the consistency of the order data
any opt This is an optional parameter that can be supplied by the workflow when an error is raised. Normally it will be a string providing additional information about the error.
Expected Return Value
NOTHING (any return value will be ignored)
Error Handler Function Definition Example
# type: GENERIC
# version: 1.0
# desc: example error handler function
%new-style
%require-types
%strict-args
%enable-all-warnings
sub example_error_handler(string err, *hash hash, any info) {
log(LL_INFO, "error handler: err=%n, hash=%n, info=%n", err, hash, info);
}
# END

Workflow Error Function

Synopsis
This function is executed by oload when the workflow is loaded into the database, therefore this function should not depend on runtime features of Qorus. It returns a hash of error information in the format described below. See Workflow Error Handling and Recovery for more information on how this function and its return value are used. See Global and Workflow-Specific Error Definitions for more information about workflow error level logic
Function Template
hash sub functionname() {}
Expected Return Value
A hash where each key is the error string code (ex: "PAYMENT-MESSAGE-TIMEOUT"), and each value is a hash with error information as given in the following description of the keys:
  • desc: the description of the error
  • severity: the severity code; see Error Severity Codes, default: OMQ::ES_Major
  • status: the status code for the step due to this error; either OMQ::StatRetry or OMQ::StatError (default if not present)
  • business: is this a business error? default: False
  • retry-delay: the default retry time for the error as an integer giving seconds (ie 1200 for 20 minutes) or a relative date/time value (i.e. 2D + 12h or P2D12H, both meaning 2 days and 12 hours); the relative date/time value format is normally recommended as it's more readable
  • level: the error level value; see Error Level Type Constants for possible values; default: OMQ::ErrLevelAuto ("AUTO")

    Only the desc key is required. Note:
    • if the "severity" key is not present, the error has a default severity of OMQ::ES_Major ("MAJOR")
    • if the "status" key is not present, the default is OMQ::StatError ("ERROR")
    • if the "level" key is not present, the default is OMQ::ErrLevelAuto ("AUTO")
    • the "retry-delay" key should only be given on errors with status set to OMQ::StatRetry ("RETRY")
Error Function Definition Example
# type: GENERIC
# version: 1.0
# desc: example error definitions
%new-style
%require-types
%strict-args
%enable-all-warnings
hash sub example_errors() {
return (
"EXAMPLE-ERROR-1": (
"desc" : "example error 1",
"status" : OMQ::StatRetry,
"retry-delay" : 45s,
"business" : False,
),
"EXAMPLE-ERROR-2": (
"desc" : "workflow-specific example error 2 (business)",
"status" : OMQ::StatRetry,
"retry-delay" : 2s,
"business" : True,
"level" : OMQ::ErrLevelWorkflow,
),
);
}
# END

Step Definitions

In the previous section we learned how to define simple steps where the step's name and version attributes were the same as that for the primary step code. However this is not enough to enable the development of complex workflows requiring more advanced functionality. Furthermore, as all workflows should be developed to provide comprehensive error recoverability in the case of errors, additional error-handling functionality is required from the system. This section will outline all the options for defining steps supporting the advanced functionality of Qorus.

The following diagram illustrates a subset of the attributes of a step that can be defined.

step.png
Step Metadata Diagram

Note that each logic attribute is defined by either a step class or a function object, and the step ID is not assigned in the workflow definition file, but rather by the loader (oload).

Class Step Definitions

The recommended way to define steps is by defining all the step's logic as a subclass of one of the following step classes:

Step Classes per Step Type

Step Type Step Class
Asynchronous Steps Qore: QorusAsyncStep
Java: QorusAsyncStep
Workflow Synchronization Event Steps Qore: QorusEventStep
Java: QorusEventStep
Normal Steps Qore: QorusNormalStep
Java: QorusNormalStep
Subworkflow Step Qore: QorusSubworkflowStep
Java: QorusSubworkflowStep
Array Asynchronous Steps Qore: QorusAsyncArrayStep
Java: QorusAsyncArrayStep
Array Workflow Synchronization Event Steps Qore: QorusEventArrayStep
Java: QorusEventArrayStep
Array Normal Steps Qore: QorusNormalArrayStep
Java: QorusNormalArrayStep
Array Subworkflow Step Qore: QorusSubworkflowArrayStep
Java: QorusSubworkflowArrayStep

In order to define class steps, each step declaration in the workflow definition file must be defined as a hash containing the following attributes:

Step Definition Hash Key Descriptions

Step Key Type Mand.? Description
name string Y The name and optionally the version of the step (i.e.: "step1:1.0")
version string N The version of the step; this key is not required in the step definition if the step's version is given in the step's name (i.e.: "step1:1.0")
desc string N The description of the step; if this key is not present, then the Qorus server will return the description of the primary step code when step information is requested
classname string N The name and optional version (i.e. "MyStep1:1.0") of the step class, providing all the logic for a step. Note that if this key is not given, oload will assume that a function-based step is being defined
classversion string N The version of the step class if not given in the "classname" parameter
queue string N The name of the queue associated with this step (corresponding to the "Message Queue" element in the diagram above). This key is required for asynchronous steps
arraytype string N For array steps, this valid must be "SERIES", in which case the step will be an array step
subworkflow bool N If this is set to True, then the step type attribute will be set to OMQ::ExecSubWorkflow making the step a subworkflow step. Steps may not be simultaneously subworkflow and asynchronous or workflow synchronization event steps
eventtype string N The name of the workflow event type for workflow synchronization event steps; in this case the step may not be a subworkflow or asynchronous step
user-interaction bool N If this is set to True, then the asynchronous step will support APIs for user interaction; can only be set on asynchronous steps; if not present the default value is False
See also
Class Definition File for information on how to define the class containing the step's logic

Class-Based Step Constructors and Static Initialization

Class-based steps can have a constructor and classes can have static initialization, but please note that if the step has configuration items, it must be instantiated by oload in order to create the step's configuration in the system. In such a case, if the constructor or static class initialization requires features that are only available at runtime in Qorus itself, the errors raised will cause step class instantiation or static class instantiation to fail.

The step constructor takes no arguments.

Class-Based Step Configuration Items

Class-based steps can also declare configuration items to allow for the behavior of the step to be modified by users at runtime using the operational web UI or the REST API.

Step configuration items are:

  1. Created by oload when loading the step; despite the fact that they are retured with an API call, the configuration is not changed at runtime
  2. Read at runtime with workflow APIs to affect the functionality of the workflow
  3. Updated using the web UI or using the REST API with PUT /api/latest/steps/{id_or_name}/config/{name}

Step configuration items are designed to allow users to affect the execution of a workflow so that changes can be made by authorized users in the UI without requiring a change to development.

Strictly Local Step Config Item

If the strictly_local flag on a step configuration item is False, then the step configuration item is not local and the value can also be set on workflow or global level.

If the strictly_local flag is False, then the step configuration item is local and hence the value for this item cannot be set on workflow either global level.

Note
All configuration item values are considered to be managed by operations by design, which means that once a configuration item has been loaded into Qorus, any changes to its value with the API are persistent and will not be overwritten by subsequent loads of the configuration item by oload.
Step configuration items must be returned through a URI path in the workflow context in order to reflect the proper value of configuration items with the strictly_local flag set to False; use the following REST API to retrieve step configuration items in the context of its declaring workflow: GET /api/latest/workflows/{id_or_name}/stepinfo/{id_or_name}/config/{name}
Qore Class-Based Step Configuration Items

Step configuration items in Qore are declared by overriding the QorusConfigurationItemProvider::getConfigItemsImpl() method in the step class. The step configuration items are registerted as part of the basic configuration of the step by oload when the workflow is loaded.

Each ConfigItemInfo hash in the return value of this method defines a configuration item for the step. The value of configuration items can then be retrieved and used in the workflow by calling one of the following APIs.

API Type API Description
Qore WorkflowApi::getConfigItemValue() retrieves a single configuration item value
Qore WorkflowApi::getConfigItemHash() retrieves all configuration items
Qore Step Config Item Declaration Example:
public class ExampleNormalStep inherits QorusNormalStep {
primary() {
log(LL_INFO, "running %s::primary()", self.className());
}
private *hash<string, hash<ConfigItemInfo>> getConfigItemsImpl() {
return {
"example-string": <ConfigItemInfo>{
"default_value": "example",
"description": "This is an example local configuration item",
"strictly_local": True,
},
"example-int": <ConfigItemInfo>{
"type": "int",
"default_value": 1,
"description": "This is an example configuration item",
},
};
}
}
Note
If the Qore-based workflow depends on a module that defines the configuration items, the workflow must be loaded with oload for the configuration items or changes to the configuration items to take effect; only oload reads the configuration items; changes to configuration items are not detected at runtime.
Java Step Class-Based Step Configuration Items

Step configuration items in Java are declared by overriding the QorusStepBase.getConfigItemsImpl() method in the step class. The step configuration items are registerted as part of the basic configuration of the step by oload when the workflow is loaded.

Each ConfigItem object in the return value of this method defines a configuration item for the step. The value of the configuration item can then be retrieved and used in the workflow by calling one of the following APIs.

API Type API Description
Java WorkflowApi.getConfigItemValue() retrieves a single configuration item value
Java WorkflowApi.getConfigItemHash() retrieves all configuration items
Java Step Config Item Declaration Example:
public class MyExampleNormalStepClass implements QorusNormalStep {
public void primary() throws Throwable {
// ... primary step logic
}
protected HashMap<String, ConfigItem> getConfigItemsImpl() {
return new HashMap<String, ConfigItem>() {
{
put("example-string", new ConfigItem("This is an example configuration item").withDefaultValue("test"));
put("example-int", new ConfigItem("This is an example configuration item").withType("int"));
}
};
}
}
Note
If the Java-based service depends on a compiled object that defines the configuration items, the service must be loaded with oload for the configuration items or changes to the configuration items to take effect; only oload reads the configuration items; changes to configuration items are not detected at runtime.

Class-Based Step User Metadata

Steps can also provide user-defined metadata which is returned as part of the step's description by overriding one of the following methods in the step class:

Note that step metadata is created by oload when the step is loaded, therefore this method should not rely on any runtime features of Qorus that are not available in oload.

Function-Based Step Definitions

For backwards compatibility, Qorus supports defining steps with functions defining each logic or code attribute of a step. In order to define complex function-based steps, each step declaration in the workflow definition file must be defined as a hash containing the following attributes:

Step Definition Hash Key Descriptions

Step Key Type Mand.? Description
name string Y The name and optionally the version of the step (i.e.: "step1:1.0")
version string N The version of the step; this key is not required in the step definition if the step's version is given in the step's name (i.e.: "step1:1.0"
desc string N The description of the step; if this key is not present, then the Qorus server will return the description of the primary step code when step information is requested
funcname string N The name and version of the primary step code (corresponding to the "Step Logic" element in the diagram above). Note that if this key is not given, the step function's name and version will be assumed to be the same as the step's (as defined in the name key)
valname string N The name and version of the validation code for the step (corresponding to the "Validation Function" element in the diagram above). The validation code is run whenever the step is recovered. Note that validation code may not be defined for subworkflow or workflow synchronization event steps
endname string N The name and version of the back-end code for asynchronous steps (corresponding to the "Async Logic" element in the diagram above). Note that if this key is defined, a queue must be defined, and the step type attribute will be set to OMQ::ExecAsync. The step may not also be a subworkflow step or a workflow synchronization event step
queue string N The name of the queue associated with this step (corresponding to the "Message Queue" element in the diagram above). This key is required for asynchronous steps
arrayname string N The name and version of the array function (corresponding to the "Array Logic" element in the diagram above). If this key is defined, then the step will be an array step
subworkflow bool N If this is set to True, then the step type attribute will be set to OMQ::ExecSubWorkflow making the step a subworkflow step. Steps may not be simultaneously subworkflow and asynchronous or workflow synchronization event steps
eventtype string N The name of the workflow event type for workflow synchronization event steps; in this case the step may not be a subworkflow or asynchronous step
user-interaction bool N If this is set to True, then the asynchronous step will support APIs for user interaction; can only be set on asynchronous steps; if not present the default value is False
Note
It's recommended to use class-based steps for the best forward compatibility with future versions of Qorus

Normal Steps

A normal step is a step that is not a subworkflow, asynchronous, or workflow synchronization event step. Normal steps do not have asynchronous back-end code, a queue, or a workflow synchronization event type, and their subworkflow attribute is False.

A normal step may be an array step, which would make it a normal array step.

The base step classes to be inherited by normal step classes are:

Qore classes:

Java classes:

See the above classes for example step definitions.

Normal Class-Based Step Definition Example

# example step definition
const simple_step = (
"name": "simple_step:1.0",
"classname": "MyNormalStep:1.0",
);
See also
class-based steps

Normal Function-Based Step Definition Example

# example step definition
const simple_step = (
"name": "simple_step:1.0",
"valname": "simple_validation:1.0",
);
Note
It's recommended to use class-based steps for the best forward compatibility with future versions of Qorus
See also
function-based steps

Subworkflow Step

A subworkflow step binds a child workflow (called a subworkflow) to a step. The child workflow's status will be bound to the step's status; that is; whatever status the child workflow has will be reflected as the step's status. This is how Qorus supports logical branching in workflows, where one branch of processing is optionally executed based on a logical condition.

The base step classes to be inherited by subworkflow step classes are as follows.

Qore classes:

Java classes:

See the above classes for example step definitions.

Subworkflow steps are defined by setting the subworkflow attribute to True in the workflow configuration file.

Subworkflow steps are not bound to any particular workflow type; the only rule for a subworkflow step is that one of the following API calls must be made in the primary step code for the steps.

Qore methods:

Java methods:

Note

A subworkflow step may be an array step, which would make it a subworkflow array step.

Note
Child workflows can provide feedback to the parent by calling WorkflowApi::leaveParentFeedback() (in Qore) or WorkflowApi.leaveParentFeedback() (in Java), and the parent can retrieve the feedback (in a step executed after the subworkflow step) by calling WorkflowApi::getChildFeedback() (in Qore) or WorkflowApi.getChildFeedback() (in Java).

Subworkflow Class-Based Step Definition Example

# example step definition
const subworkflow_step = (
"name": "subworkflow_step:1.0",
"classname": "MySubworkflowStep:1.0",
"subworkflow": True,
);
See also
class-based steps

Subworkflow Function-Based Step Definition Example

# example step definition
const subworkflow_step = (
"name": "subworkflow_step:1.0",
"subworkflow" : True,
);
Note
It's recommended to use class-based steps for the best forward compatibility with future versions of Qorus
See also
function-based steps

Asynchronous Steps

Asynchronous steps allow Qorus to efficiently process asynchronous actions. In the context of Qorus workflow processing, asynchronous actions are actions that take a significant amount of time to complete. It is not necessary to know in advance how much time the action will take to complete to define an asynchronous step.

The base step classes to be inherited by asynchronous step classes are as follows.

Qore classes:

Java classes:

See the above classes for example step definitions.

To define an asynchronous step, the step definition must define asynchronous back-end code and a queue, and one of the following API calls must be made during the execution of the primary step code.

Qore methods:

Java methods:

Note
For Qore old-style function-based steps, either wf_submit_async_key() or wf_skip_async_step() must be called.

QorusAsyncStepBase::submitAsyncKey() (Qore) and submitAsyncKey() (Java) (as well as the old-style Qore wf_submit_async_key() function) save a reference to the step's action as a unique key in the namespace of the queue; all keys submitted to a queue of the same name must be unique (this is enforced by an index in the Qorus database).

The system option qorus.async_delay determines when the system will retry the step if the step's queue has not been updated in time. If there is no validation code, the default behavior for the system will be to delete any queue data for the step, and re-run the step's primary step code. If this is not the desired behavior, then you must implement validation code to control how the system reacts to an asynchronous timeout.

When the result of the asynchronous action is available, code external to the workflow must call the REST API POST /api/latest/async-queues/{queue}?action=update using the name of the queue, the unique key in the queue identifying the step that created the action, and any data representing the result of asynchronous processing. Often this is done by implementing a Qorus service to monitor the result of asynchronous processing and updating the appropriate queue entry.

Within Qorus, the following is an example of how to call the REST API POST /api/latest/async-queues/{queue}?action=update from a Qorus service; in this example, a table in an external database is polled by checking which queue entries are still outstanding by calling GET /api/v3/async-queues/{queue}?action=qinfo and then updated with POST /api/latest/async-queues/{queue}?action=update .

Asynchronous Queue Update Example
# get list of outstanding queue entries
*list<auto> l = map int($1.queuekey), UserApi::callRestApi("GET", "async-queues/bridge/qinfo?status=" + OMQ::StatWaiting);
if (!l) {
return;
}
# check if database has been updated
AbstractTable table = UserApi::getSqlTable(ds, "rd");
hash<auto> sh = {
"columns": ("ind", "sysnmbr", "status"),
"where": {
"sysnmbr": op_ne(NULL),
"ind": op_in(l),
},
};
*hash<auto> q = table.select(sh);
# loop through result set and update "bridge" queue with key as %ind
context (q) {
slog(LL_DETAIL_1, "updating queue 'bridge': ind=%n, data=%n", %ind, %%);
hash<auto> args = {
"key": %ind,
"data": %%,
};
UserApi::callRestApi("POST", "async-queues/bridge/update", args);
}

An asynchronous step may be an array step, which would make it an asynchronous array step.

Asynchronous steps define a segment discontinuity; all normal steps leading up to the asynchronous step are in the same segment; and all normal steps executed after the asynchronous step are in another segment. Qorus workflow segments are each processed in their own thread and can process workflow order data instances independently of one another. See:

# example step definition
const async_step = (
"name": "async_step:1.0",
"classname": "MyAsyncStep:1.0",
"queue": "async_queue",
);
See also

Asynchronous Function-Based Step Definition Example

# example step definition
const async_step = (
"name": "async_step:1.0",
"valname": "async_validation:1.0",
"endname": "async_backend:1.0",
"queue": "async_queue",
);
Note
It's recommended to use class-based steps for the best forward compatibility with future versions of Qorus
See also
function-based steps

Workflow Synchronization Event Steps

Workflow synchronization event steps allow many workflow orders to synchronize their processing based on a single event. A workflow synchronization event step may be an array step, which would make it a workflow synchronization event array step.

The base step classes to be inherited for asynchronous step classes are as follows.

Qore classes:

Java classes:

See the above classes for example step definitions.

The primary step code for a workflow synchronization event step must make one of the following API calls:

The step definition must reference a workflow synchronization event type as well; event keys bound or posted are treated unique within their event type. Additionally, workflow synchronization event steps may not have validation code.

Note
See also

Workflow Synchronization Event Class-Based Step Definition Example

# example step definition
const array_eventtest = (
"name": "array_eventtest:1.0",
"classname": "MyEventStep:1.0",
"eventtype": "event-test",
);
See also
class-based steps

Workflow Synchronization Event Function-Based Step Definition Example

# example step definition
const array_eventtest = (
"name": "array_eventtest:1.0",
"eventtype": "event-test",
);
Note
It's recommended to use class-based steps for the best forward compatibility with future versions of Qorus
See also
function-based steps

Array Step

Any type of step can be an array step. An array step is any step that may need to repeat its action more than once. To define an array step, first define the normal, asynchronous, subworkflow, or workflow synchronization event step, and then define the arraytype attribute for class-based steps as "SERIES" (or define an array function for a function-based step).

The recommended way to define steps is by defining all the step's logic as a subclass of one of the following step classes:

Array Step Classes per Step Type

Step Type Base Step Class
Array Asynchronous Steps Qore: QorusAsyncArrayStep
Java: QorusAsyncArrayStep
Array Workflow Synchronization Event Steps Qore: QorusEventArrayStep
Java: QorusEventArrayStep
Array Normal Steps Qore: QorusNormalArrayStep
Java: QorusNormalArrayStep
Array Subworkflow Step Qore: QorusSubworkflowArrayStep
Java: QorusSubworkflowArrayStep

See the above classes for example step definitions.

The return value of the array code will determine how many times the step will execute, and on what data. Please note that array steps have different code signatures, as the array element is always passed to the step logic code (primary step code, validation code, and asynchronous back-end code for asynchronous steps).

Array Type attribute

The arraytype attribute may be given in a step definition to ensure that the step is defined as an array step. For class-based steps, this step attribute is mandatory for array steps, and will ensure that one of the above step classes is used as a base class for the step's class.

Example:

# example step definition for a class-based normal array step
const async_array_step = (
"name": "my_step:1.0",
"classname": "MyArrayStep:1.0",
"arraytype": "SERIES",
);

Class-Based Array Step Definition Examples

Workflow Definition File Example: Class-Based Asynchronous Array Step
# example step definition for a class-based async array step
const async_array_step = (
"name": "async_array_step:1.0",
"classname": "MyAsyncArrayStep:1.0",
"queue": "async_queue",
"arraytype": "SERIES",
);
Workflow Definition File Example: Class-Based Subworkflow Array Step
# example step definition for a class-based subworkflow array step
const subworkflow_array_step = (
"name": "subworkflow_array_step:1.0",
"classname": "MySubworkflowArrayStep:1.0",
"subworkflow": True,
"arraytype": "SERIES",
);
Workflow Definition File Example: Class-Based Workflow Synchronization Event Array Step
# example step definition for a class-based event step
const array_eventtest = (
"name": "my_event:1.0",
"classname": "MyEventArrayStep:1.0",
"eventtype": "my-event",
"arraytype": "SERIES",
);

Function-Based Array Step Definition Examples

Workflow Definition File Example: Function-Based Asynchronous Array Step
# example step definition for a function-based async array step
const async_array_step = (
"name": "async_array_step:1.0",
"valname": "async_array_validation:1.0",
"endname": "async_array_backend:1.0",
"arrayname": "async_array_function:1.0",
"queue": "async_queue",
);
Workflow Definition File Example: Function-Based Subworkflow Array Step
# example step definition for a function-based subworkflow array step
const subworkflow_array_step = (
"name": "subworkflow_array_step:1.0",
"subworkflow": True,
"arrayname": "subworkflow_array_function:1.0",
);
Workflow Definition File Example: Function-Based Workflow Synchronization Event Array Step
# example step definition for a function-based event step
const array_eventtest = (
"name": "my_event:1.0",
"eventtype": "my-event",
"arrayname": "my_event_array:1.0",
);
Note
It's recommended to use class-based steps for the best forward compatibility with future versions of Qorus

Primary Step Code

Synopsis
As the name suggests, the primary step code should contain all the logic for the step when run in OMQ::WM_Normal mode (not recovering). In the step definition the primary step code is identified by the funcname key, or, if this is not present, the name of the step is assumed to be the name of the primary step code as well.
Note
Asynchronous steps, subworkflow steps, and workflow synchronization event steps all have special requirements for API calls that must be made in the primary step code; see the documentation for each step type for more information.

Primary Method

Every step base class has an abstract primary() method where the primary step logic must be defined. See the class documentation for the specific step class for more information on requirements for the primary step method.

Primary Step Function

Qorus supports defining functions as step attributes instead of using a class for backwards-compatibility. This section describes defining a primary step function for a step.

Note
It's recommended to use class-based steps for the best forward compatibility with future versions of Qorus
Function Template For Non-Array Steps
nothing sub functionname() {}
Function Template Array Steps
nothing sub functionname(auto arrayelement) {}
Note
An argument will be passed to the step function only if the step is an array step (this will be the element corresponding to that position in the array). Otherwise no argument will be passed to the step function.
Expected Return Value
NOTHING (any return value will be ignored)

Example Normal Array Function Definition
# type: STEP
# version: 1.0
# desc: my normal array function
%new-style
%require-types
%strict-args
%enable-all-warnings
sub my_array(auto element) {
log(LL_INFO, "element = %n", element);
}
# END

Example Non-Array Subworkflow Function Definition
# type: SUBWORKFLOW
# version: 2.0
# desc: my subworkflow function
%new-style
%require-types
%strict-args
%enable-all-warnings
sub my_subworkflow() {
# if "skip-subworkflow" is set, skip subworkflow
if (WorkflApi::getPption("skip-subworkflow")) {
return;
}
# bind to MY-SUBWORKFLOW version 1.0
hash<auto> h = {
"name": "MY-SUBWORKFLOW",
"version": "1.0",
};
# create workflow data
string key = sprintf("%s-%d", WorkflowApi::getStepInfo().name,
WorkflowApi::getWorkflowInstanceData("workflow_instanceid"));
# take the static data for the subworkflow from the "subworkflow" key in
# this order's static data
hash<auto> data.staticdata = WorkflowApi::getStaticData().subworkflow;
hash<auto> r = wf_bind_subworkflow(h, data);
log(LL_INFO, "subworkflow created, workflow_instance_id=%d", r.workflow_instanceid);
}
# END

Example Non-Array Workflow Synchronization Event Function Definition
# type: STEP
# version: 1.0
# name: my_event
# desc: my event step function
%new-style
%require-types
%strict-args
%enable-all-warnings
sub my_event() {
# sip the event if the "skip-event" option is set
if (WorkflowApi::getOption("skip-event") {
return;
}
# get an event from my event service
string event = omqservice.user.myevent.get();
wf_bind_event(event);
}
# END

see $OMQ_DIR/examples/user/TEST-WORKFLOWS/test-functions-v1.0.qfd for further example function definitions for all function types in this section.

Validation Code

Synopsis
The validation code is run whenever the step is recovered (for asynchronous steps, this could also be if the asynchronous step's queue entry has not been updated within the time period defined by the qorus.async_delay system option). This function's return value tells Qorus if the step function should be run again or not. Note that subworkflow steps and workflow synchronization event steps cannot have validation code.
Note: Always Implement Validation Code for Non-Repeatable Actions
A number of problems could prohibit a step from being updated with the correct status, for example, network problems, power outages, other application problems, etc. To handle these situations gracefully, implement validation code to verify the status of the step before running the step's primary step code again. The arguments passed to the validation code depend on the type of step.

Validation Code Return Value

The following table describes how the system reacts depending on the return value of the validation logic.

Validation Code Return Value

Return Value System Behavior
OMQ::StatComplete Do not run the primary step logic; mark the step as "COMPLETE" and continue. For asynchronous steps, back-end code also will not be run
OMQ::StatError Do not run the primary step logic; mark the step as "ERROR" and stop running any further dependencies of this step
OMQ::StatRetry Run the primary step loggic again immediately. If the step is an asynchronous step with queue data with a OMQ::QS_Waiting status, the queue data will be deleted immediately before the primary step logic is run again
OMQ::StatAsyncWaiting For asynchronous steps only, do not run the primary step logic and keep the "ASYNC-WAITING" status. For non-asynchronous steps, raises an error and the return value is treated like OMQ::StatError
any other status an error is raised and the return value is treated like OMQ::StatError

Validation Method

Some step base classes have a validation() method that can be overridden where the validation logic can be defined. See the class documentation for the specific step class for more information on the signature and requirements for the validation method (if it's supported for the step type, not all step types support validation logic), and see Validation Code Return Value for a description of how the return value of this method affects workflow order processing.

Validation Step Function

Qorus supports defining functions as step attributes instead of using a class for backwards-compatibility. This section describes defining a validation function for a step.

Note
It's recommended to use class-based steps for the best forward compatibility with future versions of Qorus
Function Template For Non-Asynchronous Non-Array Steps
string sub functionname() {}
Function Template For Non-Asynchronous Array Steps
string sub functionname(auto array_element) {}
Function Template For Asynchronous Array Steps
string sub functionname(*string async_key, auto array_element) {}
Function Template For Asynchronous Non-Array Steps
string sub functionname(*string async_key) {}
Expected Return Value
string: step status constant

Example Asynchronous Validation Function
# type: VALIDATION
# version: 1.0
# desc: my aysnc validation function
%new-style
%require-types
%strict-args
%enable-all-warnings
string sub my_async_validation(*string key) {
# call check_action() to check in the target DB if the action has completed
if (check_action())
return OMQ::StatComplete;
# call check_pending() to see if we retry or still wait
return check_pending() ? OMQ::StatAsyncWaiting : OMQ::StatRetry;
}
# END
Note
Asynchronous steps should normally have validation code, because Qorus will automatically try to recover an asynchronous step if the queue is not updated within the time period defined by the qorus.async_delay system option. If no validation code is defined, then the system will automatically delete the queue data (if any is present) and rerun the primary step code, which may not be the desired behavior.
Subworkflow steps will never be recovered in the parent workflow, and subworkflow steps may not have validation code for this reason. A subworkflow step's status is bound to the subworkflow, and any errors must be corrected in the subworkflow.
Workflow synchronization event steps cannot have validation code; they are either waiting for their event or OMQ::StatComplete. To force a Workflow synchronization event step to continue, either post its event or call omq.system.skip-step() on the step.

Asynchronous Back-End Code

Synopsis
This code must be defined for asynchronous steps. This logic is run when the queue data bound to the asynchronous step is updated and receives the status OMQ::QS_Received. The job of this code is to determine if the asynchronously-received data is correct or not; if it is not correct, then the code should raise an error by calling WorkflowApi::stepError()() (in Qore code) or WorkflowApi.stepError() (in Java code) or by throwing an appropriate exception.

Asynchronous Back-End Method

Asynchronous steps can define an end() method to process data submitted for the asynchronous workflow event. See the class documentation for the specific step class for more information on the end() method.

Asynchronous Back-End Step Function

Qorus supports defining functions as step attributes instead of using a class for backwards-compatibility. This section describes defining a asynchronous back-end function for a step.

Note
It's recommended to use class-based steps for the best forward compatibility with future versions of Qorus
Non-Array Backend Function Template
nothing sub functionname(auto queue_data) {}
Array Backend Function Template
nothing sub functionname(auto queue_data, auto array_element) {}
Note
The array_element argument will be passed to the back end function only if the step is an array step. Otherwise only the queue_data argument will be passed to the function; the queue_data argument is the data that is posted on the queue when with the POST /api/latest/async-queues/{queue}?action=update call when the queue is updated.
Expected Return Value
NOTHING (any return value will be ignored)

Example back-end code
# type: ASYNC-END
# version: 1.0
# desc: my async back end function
%new-style
%require-types
%strict-args
%enable-all-warnings
sub my_async_end(hash<auto> data) {
# the error thrown here should be defined by the workflow's errorfunction
if (data.status == "ERROR")
throw "ASYNC-RECEIVE-ERROR";
else
log(LL_INFO, "data status: %y", data.status);
}
# END

Queue Tag

Synopsis
The queue tag must be set to a string giving the name of the queue for asynchronous steps. The queue will link the asynchronous step's primary step code with the back-end code.

When the result of the asynchronous action is available, the result must be posted to the step's queue using the key created in the step's primary step code when the QorusAsyncStepBase::submitAsyncKey() method (for Qore class-based steps) or the submitAsyncKey() method (for Java) is called. The queue will hold this data (stored in the database table QUEUE_DATA) and pass it to the asynchronous back-end code in order to determine the step's status.

See Queue Definitions for more information.

Array Code

Synopsis
This code is run every time Qorus starts executing an array step. This could be in normal mode (the first time the step is executed), in recovery mode, or when Qorus attaches to a workflow order data instance in order to execute back-end code. For each element in the list returned, Qorus will run the appropriate function and track the results separately. If no value or an empty list is returned the first time the array function is executed, the entire array step receives a COMPLETE status and the step function is not run.
Note
The return value of the array code is not stored in persistent storage. In order to ensure correct functionality in the case of recoveries (or with back end logic in asynchronous steps), the array function must return the same array with elements in the same order every time it is run. If necessary, the list or information to regenerate the list can be stored in dynamic order or dynamic step data.

Array Method

For class-based steps, the array method is an abstract method defined in all array base step classes and must be defined in the step's class.

Array Function

Qorus supports defining functions as step attributes instead of using a class for backwards-compatibility. This section describes defining a asynchronous back-end function for a step.

Note
It's recommended to use class-based steps for the best forward compatibility with future versions of Qorus
Function Template
any sub functionname() {}
Possible Return Values
list: List of elements for array step; if the list is empty, then the array step is skipped
any: A single element to execute a single substep in the array step
nothing: if NOTHING is returned, then the array step is skipped

Example of Array Function Definition
# type: ARRAY
# version: 1.0
# desc: my array function
%new-style
%require-types
%strict-args
%enable-all-warnings
*list sub my_array() {
# returns the list under the AccountInfoList key in static data
return WorkflowApi::getStaticData().AccountInfoList;
}
# END

Subworkflow Tag (subworkflow)

Synopsis
If this boolean tag is set to True, then the step will be a subworkflow step, and the step's status will be bound to a new workflow (the subworkflow) when the QorusSubworkflowStepBase::bindSubworkflow() method (in Qore class-based steps) or the QorusSubworkflowStepBase.bindSubworkflow() method (in Java) is called. If neither this method nor QorusSubworkflowStepBase::skipSubworkflow() (in Qore class-based steps) nor QorusSubworkflowStepBase.skipSubworkflow() (in Java) is called, then the step will exit with an error. Otherwise, if a subworkflow is bound, the step will receive a WAITING status until the subworkflow is COMPLETE or ERROR, at which time the step will be updated with the subworkflow's status.
See also
Subworkflow Step for an example of a subworkflow step definition where this key is set to True
Note: Use a Subworkflow Step for Logical Branching
Each step in a Qorus workflow is executed and must receive a COMPLETE status in order for the workflow to become COMPLETE. If there is a series of steps that is only conditionally executed, bind it to a subworkflow step and create a workflow that will be bound to that step.

Event Type Tag (eventtype)

Synopsis
If this string tag is present, then the step will be a workflow synchronization event step, and the step must either be bound to a workflow synchronization event by calling QorusEventStepBase::bindEvent() or QorusEventStepBase::bindEventUnposted() (in Qore class-based steps) or QorusEventStepBase.bindEvent() or QorusEventStepBase.bindEventUnposted() (in Java), or skipped by calling QorusEventStepBase::skipEvent() (in Qore class-based steps) or QorusEventStepBase.skipEvent() (in Java) in the step's primary step code. If none of these APIs are called in the step's primary step code, the step will exit with an error. Otherwise, if a workflow synchronization event is bound to the step, the step will only receive a COMPLETE status when the event is posted by calling UserApi::postSyncEvent(), PUT /api/latest/sync-events/{type}/{key}?action=post, or omq.system.post-event().

The value of the tag must be the name of a valid workflow synchronization event.
See also
Workflow Synchronization Event Steps for an example of a workflow synchronization step definition where this key is defined.

Asynchronous User Interaction Tag

Synopsis
If this is step key is set to True, then the asynchronous step will support APIs for user interaction; can only be set on asynchronous steps. If this key is not present in the step definition, the default value is False.
REST API support
The following APIs can be used to atomically acquire step data for editing by a particular user:
See also
asyncsteps

Function Definition File

Qorus function definition files should have the extension: *.qfd

Function definition files are used to create function objects in the Qorus schema that can then be used by workflows (or loaded into service program objects as well for GENERIC functions).

Functions are defined by adding metadata tags in the form of special line comments to normal text files containing function definitions. The tags are parsed by the oload program, which will create function objects in the Qorus database according to the definitions in the file.

Function definitions begin with metadata tags as given in as specially-formatted comments as seen in the example below. After the header tags, one or more function definitions then follow, and the END tag terminates the function definition.

Function Definition File Tags

Tag Mand.? Description
type Y must be the first tag in a function definition, value must be one of the Step Function Types
name N If the name tag is not defined, then the name of the function object in the database will be taken from the last function defined in the code block. If the name tag is defined, then a function with this name generally must be defined in the code block when used as a step function, however when used as a library function the name of the function object does not have to correspond to any function in the code block
version Y Version string for the function object
desc Y Description of the function object
author N The optional author of the code object
patch N Optional patch string – should be updated every time the same version of the function object is updated in the database
TAG N Option tag definition in the format "key: value"
END Y This tag must be included to terminate the function definition
Note
A single function object can contain more than one function definition, but must contain at least one function definition with the same name as the object, unless if the function object will be used as a library function, then the name of the function object does not have to correspond to any function in the code block.
Example Function Definitions
Here is an example function definition with the minimum required tags:
# type: GENERIC
# version: 1.0
# desc: example function for the Qorus Developer's Guide
# TAG: example-code: true
%new-style
%require-types
%strict-args
%enable-all-warnings
sub example_func() {
log(OMQ::LL_INFO, "the example function is here");
}
# END

Here are further examples:

Class Definition File

Qorus class definition files should have the extension: *.qclass or *.qclass.java (for Java-based classes).

Class definition files are used to create class objects in the Qorus schema that can then be used by workflows (or loaded into service program objects as well).

Like function objects, classes are defined by adding metadata tags in the form of special comments to normal text files containing class definitions. The tags are parsed by the oload program, which will create class objects in the Qorus database according to the definitions in the file.

Class definitions begin with metadata tags as given in as specially-formatted comments as seen in the examples below. After the header tags, one or more class definitions then follow, and the END tag terminates the class definition.

Class Definition File Tags

Key Mand.? Description
name N If the name tag is not defined, then the name of the class object will be taken from the last class defined in the code block. If the name tag is defined, then a class with this name should normally be defined in the code block, but the current version of Qorus does not enforce this
version Y Version string for the class object
desc Y Description of the class object
author N The optional author of the code object
patch N Optional patch string – should be updated every time the same version of the class object is updated in the database
lang N the possible values are "qore" for Qore code (the default) and "java" for code targeting the Java 8 JVM (see Developing in Java)
requires N Optional list of classes required by the current class; causes the required classes to be loaded when the current class is loaded
TAG N Option tag definition in the format "key: value"; for Java classes, entries in the classpath can be added here by adding a colon-separated list of path names to the "classpath" tag; note also that $OMQ_DIR (or any other server-side environment variable) can be used in classpath entries and will be resolved to the Qorus directory
END Y This tag must be included to terminate the class definition
Example Qore Class Object Definition (QoreExampleClass-v1.0.qclass)
# name: QoreExampleClass
# version: 1.0
# desc: example class for the Qorus Developer's Guide
# patch: p1
# TAG: example-code: true
%new-style
%require-types
%strict-args
%enable-all-warnings
class QoreExampleClass {
constructor() {
log(LL_INFO, "QoreExampleClass::constructor() here");
}
}
# END

Example Java Class Object Definition (JavaExampleClass-v1.0.qclass.java)
// name: JavaExampleClass
// version: 1.0
// desc: example class for the Qorus Developer's Guide
// patch: p1
// lang: java
// TAG: example-code: true
// TAG: classpath: $OMQ_DIR/user/jar/my-jar-1.jar:$OMQ_DIR/user/jar/my-jar-2.jar
package org.example;
// import MyClass from my-jar.jar
import org.example.MyClass;
// inherit UserApi so static members and methods can be used without a class prefix
class JavaExampleClass extends UserApi {
JavaExampleClass() {
log(LL_INFO, "JavaExampleClass constructor() here");
}
}
// END

Note
  • A single class Qore object definition can contain more than one class definition, but by convention should normally contain at least one class with the same name as the object.
  • Java class definitions must contain only one class, and the class defined must correspond to the name or Java compilation will fail
  • The classpath tag can be used to add entries to the classpath for Java classes; $OMQ_DIR can be used in classpath entries as in the above example and will be resolved to the Qorus directory; see Java Classpath Handling in Qorus for more information
See also
the classes listed in Class Step Definitions for example class definitions for class-based steps

Constant Definition File

Qorus constant definition files should have the extension: *.qconst

Constant definition files are used to create constant objects in the Qorus schema that can then be used by workflows (or loaded into service program objects).

Like function and class objects, constant objects are defined by adding metadata tags in the form of special comments to normal text files containing constant definitions. The tags are parsed by the oload program, which will create constant objects in the Qorus database according to the definitions in the file.

Constant definitions begin with metadata tags as given in as specially-formatted comments as seen in the example below. After the header tags, one or more constant definitions then follow, and the END tag terminates the constant definition.

Constant Definition File Tags

Key Mand.? Description
name Y The name tag must be defined for constant objects
version Y Version string for the constant object
desc Y Description of the constant object
author N The optional author of the code object
patch N Optional patch string – should be updated every time the same version of the constant object is updated in the database
TAG N Option tag definition in the format "key: value"
END Y This tag must be included to terminate the constant definition
Example Constant Object Definition
# name: exampleConstant
# version: 1.0
# desc: example constant for Qorus Developer's Guide
# patch: p1
# TAG: example-code: true
%new-style
%require-types
%strict-args
%enable-all-warnings
namespace Test {
const this = 1;
const that = "two";
}
# END