Qorus Integration Engine®  4.0.3.p2_git
System Architecture

Introduction

Qorus Integration Engine® provides a framework for the reliable operation of business-critical interfaces. From a high level, the Qorus Integration Engine® server is made up of a UNIX process tightly-bound to an Oracle, PostgreSQL, or MySQL database, communicating with other systems and clients through the network or the enterprise message bus as follows:

Qorus External Architecture Overview-1.png
Qorus High Level External Architecture

System Architecture Overview

The following figure gives a high-level overview of the internal system architecture:

system-architecture7.png
Qorus Integration Engine® System Architecture Overview

The basis for the system is the Qore programming language; large parts of the Qorus system itself and all user code is written in this programming language.

The Qore programming language provides features that enable embeddable user logic, automatic and explicit character set encoding conversions, threading capabilities and thread synchronization primitives, as well as object classes that provide interfaces to other systems, such as Oracle, Sybase, MySQL, PostgreSQL, Microsoft SQL Server, TIBCO Rendezvous® and TIBCO Adapter™ components, web service and other communication protocols, and more.

The Qorus system provides a workflow logic cache, a workflow API, and a workflow execution engine, a service loader, version-aware service logic cache, a service API, an RBAC system API, and a multithreaded HTTP server servicing YAML-RPC, XML-RPC, JSON-RPC, and HTTP requests, among others. Because the YAML-RPC, XML-RPC, and JSON-RPC handlers export all service methods automatically as well as built-in system API methods, as web service calls system and user services can be considered run-time extensible and run-time upgradeable API sets.

The APIs made available by the Qorus system are listed in the following table.

Qorus APIs

API Type Description
Workflow API Internal Set of functions, objects, classes, etc exported in the Qore Programming Language for use by workflow code.
Service API Internal Set of functions, objects, classes, etc exported in the Qore Programming Language for use by system or user service code
Job API Internal Set of functions, objects, classes, etc in the Qore Programming Language for use by job code
Mapper API Internal Set of functions, objects, classes, etc in the Qore Programming Language for use by mapper code
System Service API Internal and External Set of services delivered with the system providing additional functionality to Qorus
RPC System API External The RPC network API made available to any client on the network through the YAML-RPC, XML-RPC, and JSON-RPC protocols (and optionally through TIBCO Rendezvous)
Note
Service methods are exported through the system API by default. Because these methods are available both internally and externally, they can be verified and tested separately from dependent workflows and/or services. System and exported service methods can also be easily called from the Windows GUI or from the command-line (using the ocmd program) or other programs using the YAML-RPC, XML-RPC, or JSON-RPC protocols.

Qorus relies on its system schema to store most of its configuration information, some system logic (in system services), and all user configuration.

The following table outlines the data that is stored in the system database.

Qorus Database Configuration

Data Description
Workflow Metadata The description of a workflow (name, version, steps, dependencies, etc) and all workflow logic written in the Qore programming language using the workflow API
Service Metadata The description of a system or user service (method names, version, etc) and service method logic written in the Qore programming language and using the service API
Job Metadata The descripion of time-based jobs and job logic written in the Qore programming language and using the job API
Workflow Order Data Workflow order data instances, the data and processing status

Qorus is a highly multithreaded workflow engine that can be considered a workflow and service manager. Due to the fact that much of the system code is stored in a database, and can be "refreshed" (i.e. old version deleted from the cache, new version read in from the database and started) while the system is running, not only can use logic be upgraded without shutting the system down, parts of the workflow system itself (system services) can also be upgraded live without shutting the system down.

The majority of the additional programs included with Qorus are client programs to the workflow engine. They communicate with the Qorus server via the HTTP server using the YAML-RPC, XML-RPC, or JSON-RPC protocols.

Qorus features a built-in multithreaded HTTP server which functions as the system's interface to the outside world.

All system activities will happen within a single UNIX process that manages many concurrent threads to get its work done.

Qorus Directory Layout

For self-contained application installations; the Qorus Integration Engine® application directory has a structure as given in the following table (see below for LSB file locations, such as when Qorus is installed with an RPM package on Linux, LSB stands for Linux Standard Base).

Qorus Integration Engine® Self-Contained Application Directory Layout

Directory Description
$OMQ_DIR/bin System binaries are located here
$OMQ_DIR/docs System documentation can be found here
$OMQ_DIR/etc System configuration files
$OMQ_DIR/examples/client-lib Example programs using the qore language client-library
$OMQ_DIR/examples/qore Example programs in the qore language
$OMQ_DIR/examples/user Example Qorus user code definitions
$OMQ_DIR/lib System and included third-party library files
$OMQ_DIR/qlib Qore-language include file directory
$OMQ_DIR/modules System modules for optional features, loaded on demand
$OMQ_DIR/modules/auto System modules for features loaded at start
$OMQ_DIR/releases System load files
$OMQ_DIR/sql System schema files
$OMQ_DIR/system System service definitions
$OMQ_DIR/templates Miscellaneous files
$OMQ_DIR/user Qorus user definition files, code objects, interface descriptions, release source

Qorus Integration Engine® LSB Application File and Directory Locations

Directory Description
/etc/qorus/ Application configuration files: System Options File
/etc/sysconfig/qorus Application configuration file for runlevel script
/etc/init.d/qorus Runlevel script for starting and stopping the application
/usr/bin Directory where all system executables are stored
/usr/lib (or /usr/lib64 for 64-bit) Directory where the Qore library and binary modules are stored
/var/log/qorus Directory where all application log files will be written by default
/usr/share/doc/packages/qorus Directory where Qorus documentation will be stored
/usr/share/doc/packages/qore Directory where all Qore documentation will be stored if the qore-doc package is installed
/usr/include/qore Directory where all Qore include files will be stored if the qore-devel package is installed
/var/opt/qorus Directory where all other application files are stored (see below for details)
/var/opt/qorus/examples/client-lib Example programs using the qore language client-library
/var/opt/qorus/examples/qore Example programs in the qore language
/var/opt/qorus/examples/user Example Qorus user code definitions
/var/opt/qorus/qlib Qore-language include file directory
/var/opt/qorus/modules System modules for optional features, loaded on demand
/var/opt/qorus/modules/auto System modules for features loaded at start
/var/opt/qorus/releases System load files
/var/opt/qorus/sql System schema files
/var/opt/qorus/system System service definitions
/var/opt/qorus/templates Miscellaneous files
/var/opt/qorus/user Qorus user definition files, code objects, interface descriptions, release source

Configuration Files

Contents of this section:

System Options File

The options file contains system and client options. The location of the file depends on the installation type:

  • $OMQ_DIR/etc/options for tar installations
  • /etc/qorus/options for LSB installations

Qorus System options have the format:

  • qorus.option = value

Example:

qorus.auto-recover = true 

Also a colon may be used instead of an = as in:

qorus.auto-recover: true 

Any text after a # character is assumed to be a comment.

The most important option that should be set before the system will start is instance-key. This must be a unique identifier for the instance that will enable the system to avoid accidentally starting the same instance more than once and will also allow the system to recover the application session gracefully errors should the system ever terminate abnormally. The default value for this key is "qorus-test-instance".

Example qorus.instance-key option setting:

qorus.instance-key: bss-uat-1 

Additionally, if the log files for Qorus should be written to a directory other than the default (the default for LSB installations is /var/log/qorus, the default for tar installations is $OMQ_DIR/log), the qorus.logdir option must be set to the directory where Qorus log files will be written. This directory must be writable by the user running the Qorus server:

Example qorus.logdir option setting:

qorus.logdir: /var/log/qorus 

Another key of interest in the initial installation is http-server:

Example qorus.http-server option setting:

qorus.http-server: localhost:8081,192.168.20.4:9010 

The default for the HTTP server value is 8001, meaning that the application will listen on all interfaces on port 8001.

See also
System Options for a complete reference to all system options.

Qorus client options have the format:

  • qorus-client.option = value

Example:

qorus-client.proxy-url = http://user:pass@192.168.25.7:8080 
See also
OMQ::client_option_hash for possible client options.

Obsolete System Files Migrated to the Database

Upgrade Migration Info: System Datasource (dbparams) File

As of Qorus 4.0 the dbparams file is no longer used by Qorus; all system datasources are handled as datasource connection objects instead.

When upgrading a per-4.0 version of Qorus, datasources defined in any existing dbparams file are automatically migrated to the system database and created as datasource connection objects.

See also
Datasource Connections

Upgrade Migration Info: Remote Qorus Instance Configuration File

As of Qorus 4.0 the remoteconnections file is no longer used by Qorus; all Qorus remote connections are handled as Qorus remote connection objects instead.

When upgrading a per-4.0 version of Qorus, Qorus remote connections defined in any existing remoteconnections file are automatically migrated to the system database and created as Qorus remote connection objects.

See also
Qorus to Qorus Connections

System Startup

Once launched, the system initializes itself with the following steps:


Step 1: Parse Options File

Parse System Options File; if any syntax errors are found, information is displayed and the program exits.

See also
System Options for detailed descriptions of all system options.

Step 2: Open System Datasources

Open System Datasource "omq". If the system "omq" datasource cannot be opened, then error information is displayed and the program exits.


Step 3: Open Log Files

If the log files cannot be opened, an error message is displayed and the program exits.


Step 4: Initialize RBAC Framework

users, roles, permissions, and interface groups are read in from the database. If RBAC security is enabled and no single user has both OMQ::QR_LOGIN and OMQ::QR_SHUTDOWN permissions, an error message is displayed and the system will refuse to start.


Step 5: Open Application Instance Session

If any errors are encountered, information is logged, and the program exits. The session is recovered here if system option qorus.auto-recover is True.

See also
Application Session Model for more information about opening an application session and session recovery.

Step 6: Initialize Services

Services with the autostart flag set are started automatically. Any errors auto-starting services are logged, but do not otherwise affect system startup.


Step 7: Start HTTP Server

If any errors are encountered, information is logged, any loaded services are deleted, and the program exits.


When the HTTP server is started, listeners for each entry in the System Options File (or as overridden on the command-line) will listen for requests on their respective interfaces and ports. To start and stop workflows and services, to shut down the system, to change options, etc, the appropriate system API calls must be made, normally as properly formatted REST, YAML-RPC, XML-RPC, or JSON-RPC commands sent to Qorus' HTTP server. For more details, please see HTTP Server.

System startup raises the following system event: SYSTEM_STARTUP

System Shutdown

If the shutdown message is received, the system performs the following steps:


Step 1: Stop All Workflows

All running workflow execution instances are stopped. While system shutdown is in progress, it is not possible to start new workflow execution instances.


Step 2: Stop HTTP Server

Any requests in progress must complete before the server can be stopped. After this point, no external communication is possible with the Qorus system.


Step 3: Stop and Unload All Services

All system and user services are stopped and unloaded from the service cache.


Step 4: Close Application Session

The application session is marked as closed in the system database.


Step 5: Close System Log Files

System log files are closed.


System shutdown is the last system event raised by the system; the event raised is SYSTEM_SHUTDOWN.

Limitations of Multiple Instances Sharing the Same Schema

If more than one application instance is started on the same database schema using different instance keys, there are some important limitations to be aware of.

Note
Starting more than one application instance on the same database schema using the same instance key by misusing the qorus.auto-recover option is an error will lead to unpredictable results including data corruption (incorrect statuses in the database) and other errors.

Running the Same Workflow On Multiple Instances
Qorus instances do not communicate with each other and therefore do not share workflow cache information. This has important implications for running workflows and processing asynchronous messages on multiple Qorus instances sharing the same database schema.

If an asynchronous message is delivered to a Qorus instance where the target workflow is not running (but is running in another instance on the same schema), the message will be saved to the database, but the instance(s) running the workflow will not see the change, and the message will not be processed until the workflows are reset or restarted. In this case, the asynchronous steps affected also cannot be recovered with the asynchronous recovery mechanism without restarting or resetting the workflow because the data remains the database but is never posted to the internal queue on the correct instance.

If an asynchronous message is delivered to a Qorus instance where there are running execution instances of the target workflow, but the target workflow order data instance is cached by another Qorus instance running on the same schema, an error message will be logged and the message will be discarded. In this case however, a properly designed and implemented workflow will still be able to recover (although it will take much longer than if the message had not been discarded); when the front-end segment is run in recovery mode, the validation logic should detect the lost message and enable the workflow execution instance to continue processing the workflow order data instance.

Running the Same Service On Multiple Instances If a service should only be started once (for example, it uses unique network resources), Qorus cannot prevent it from being (attempted to be) started on other Qorus instances due to the fact that there is no synchronization between Qorus instances on the same (or different) schemas.

This situation can lead to error messages in the log files (particularly if such a service is flagged with autostart=true), and can possibly lead to workflow errors as well, if workflows on that instance rely on the functionality of the service that cannot start.

No Qorus system services fall into this category.

Note that these limitations may be addressed in a future version of Qorus Integration Engine® in order to support load-sharing depending on user demand.

Role Based Access Control

Qorus Integration Engine® supports a security model based on Role Based Access Control (RBAC). In this model, users are assigned roles that have permissions associated with them that allow certain actions or types of actions to be performed in the system.

Qorus security is disabled by default, it is enabled by setting the qorus.rbac-security option to True in the Qorus System Options File. Also see the qorus.rbac-force-user option.

RBAC Concepts

RBAC Concept Description
user An authorized login to the system
role A security configuration that can be associated to users
permission Permissions are associated with roles that allow actions to be performed on the system
interface group An interface group contains lists of workflows, services, jobs, mappers, and value maps; associated interfaces can be enabled or disabled for operational reasons by enabling or disabling the interface group, and the group can be applied to roles to limit access to those objects through the system API.

When RBAC security is disabled, no authorization or authentication is necessary to call any functionality in Qorus. When RBAC is disabled, unauthenticated users have all possible user and system permissions.

When RBAC security is enabled, only users with a valid username and password may connect to the Qorus HTTP server; i.e. HTTP authentication is required; the HTTP username is the Qorus RBAC user, and the password is the user's password. Each user will have a set of assigned roles; the roles assigned determine what functionality is available to the user.

If a certain action requires one or more permissions, and the user does not have any roles with the required permission or permissions, an AUTHORIZATION-ERROR will be returned to the caller and the action will not be performed.

If a user does not have a role with the DEFAULT group, then access to workflow, services, jobs, mappers, and value maps (including associated configuration information and data) will be restricted to the objects in the interface groups that the user's roles have. In this case, calls to API methods will filter out any workflows, services, jobs, mappers, and value maps not in the interface group list, and any attempt to directly access an object (including configuration and data) for objects not in the interface group list will result in a WORKFLOW-ACCESS-ERROR, SERVICE-ACCESS-ERROR, JOB-ACCESS-ERROR, MAPPER-ACCESS-ERROR, or VALUEMAP-ACCESS-ERROR exception being thrown.

See also
RBAC Permissions for a list of system permissions and descriptions.
RBAC Methods and Access Group Methods for more information on system API methods related to RBAC functionality.
RBAC Tables for a list of RBAC tables in the Qorus system schema
Interface Groups for more information on interface groups

RBAC security only applies to communication through the Qorus HTTP server. The following components of Qorus are not subject to RBAC security because they use direct database access (and therefore can be used when the Qorus server is offline but the database is up):

Qorus Tools/Components Not Subject to RBAC Security

Tool/Component Description
oload Qorus Integration Engine® system database object manager and loader
user-tool Command-line, offline (database-only) RBAC manager (create, change, and delete users, roles, permissions, and interface groups)
schema-tool Qorus Integration Engine® schema manager
Note
RBAC security is normally not applied to internal API calls (unless explicitly applied with callNetworkAPIArgsWithAuthentication() for example)

RBAC Users

Each user has a name, password, and a list of roles. Users are identified by name and do not have an ID. A user's roles determine its permissions and interface groups.

A particular user's permissions and interface groups are calculated by combining all the permissions and interface groups for all of its roles.

If a user has at least one role with the DEFAULT interface group, then it will have unrestricted access to all workflows, services, and jobs.

RBAC Roles

Roles represent security configurations used to provide capabilities to users. They define a set of permissions and a list of interface groups to be used when creating new users. Roles may contain any number of system and/or user permissions and interface groups associated to them.

The following system roles are delivered with the system:

Note
Qorus is delivered with a set of default roles that are updated in each schema upgrade. System roles should not be changed since any changes will be overridden in the next schema upgrade or schema alignment operation.

RBAC Permissions

Permissions are associated to roles and enable actions or class of actions to be performed in Qorus. System permissions pertain to actions in Qorus, user permissions are only checked by Qorus user code. System permissions cannot be changed to deleted. Permission names are unique; a user permission cannot have the same name as a system permission and vice-versa.

See also
RBAC Permissions for a list of all system permissions supported by Qorus.

Users permissions can be created and access can be checked manually in user services by calling one of the following functions:

Interface Groups

An interface group is contains a list of workflows, user services, jobs, mappers, and value maps; groups can be enabled and disabled, and roles can have a list of groups they can access.

Interface groups service the following functions:

  • they allow larger Qorus installations the flexibility to restrict access only to data and services needed by particular operations or business personnel (ie permit a multi-tenant configuration of Qorus)
  • they allow interfaces to be grouped logically
  • they allow interfaces to be temporarily enabled or disabled for operational reasons

If a group is disabled, then no workflows, services, or jobs in the group can be started; if a group is disabled while member workflow execution instances, services are running or loaded, or jobs are active, then they will be immediately stopped when the group is disabled.

When an interface group is enabled, then all workflows with a non-zero autostart value are started, all services with their autostart flag set are loaded, and all active jobs in the group are reactivated (assuming they are not also members of another disabled group or they do not have connection dependencies that prohib them to be started).

Note
  • Access to system services cannot be restricted through membership in interface groups; access to system services can only be controlled through granting or removing the permissions OMQ::QR_CALL_SYSTEM_SERVICES_RO and OMQ::QR_CALL_SYSTEM_SERVICES_RW from a user
  • The "DEFAULT" group cannot be disabled or deleted

DEFAULT Interface Group

The "DEFAULT" group (OMQ::DefaultGroupName) with ID 0 (OMQ::DefaultGroupID) is a special group in that it automatically grants access to all workflows, services, jobs, mappers, and value maps.

By default Qorus roles have access to the "DEFAULT" group, so that all roles provided with Qorus provide unrestricted access to all interfaces in Qorus.

LDAP Authentication and RBAC Integration

Qorus can authenticate and retrieve user and role information from an LDAP server using the qorus.rbac-external option with the "QorusLdapAuth" argument.

In this case additional options need to be set the options file as in the following table.

LDAP Options

Option Mandatory? Default Example Description
ldap.uri Yes n/a ldap.uri: ldaps://localhost:1389 Sets the URI for the LDAP server (use "ldap" for non-encrypted connections, "ldaps" for encrypted connections)
ldap.base Yes n/a ldap.base: dc=example,dc=com Sets the base DN for searches
ldap.directbindprefix No n/a domain-name\ This string will be prefixed to any user name given for the bind DN
ldap.directusername No False True If True then the username given will be used for authentication and lookups against the LDAP server instead of creating a bind DN
ldap.roleattr No n/a orclmemberof The user attribute giving the roles the user has
ldap.roleregex No n/a ^cn=([-a-z0-9_]+) The regular expression extraction pattern for extracting the roles from the roleattr attribute; note that a caseless match is performed by using RE_Caseless in the internal call to regex_extract()
ldap.ubase No "ou=people" ldap.ubase: ou=staff Sets the user base for user information, this and the ldap.base argument will be used to create the bind DN
ldap.uentry No "uid" ldap.uentry: cn Sets the attribute to be used to identify users

Simple example option configuration for LDAP authentication:

qorus.rbac-security: true
qorus.rbac-external: QorusLdapAuth
ldap.uri: ldap://localhost:1389
ldap.base: dc=net,dc=internal

More complex example option configuration for LDAP authentication with Active Directory:

ldap.uri: ldap://192.168.231.18
ldap.base: dc=enterprise,dc=local
ldap.directusername: true
ldap.directbindprefix: enterprise\
ldap.roleattr: memberof
ldap.roleregex: ^cn=([-a-z0-9_\.]+)
ldap.uentry: samaccountname

System Auditing

Qorus supports writing out database records for system and user events; this solution is called system auditing; records are written to the AUDIT_EVENTS table.

System auditing is enabled by setting the qorus.audit system option. Audit events can be used to keep a security record of actions taken in the system, or to explicitly track KPIs (Key Process Indicators) for SLA tracking, for example (see the audit_user_event() function, for example).

The qorus.audit system option accepts the codes in the following table (corresponding to Audit Options).

Audit Option Codes

Code Description
alerts Setting this option will cause alert events to be written to the AUDIT_EVENTS table
api Setting this option will cause every api call that makes changes to be audited in the AUDIT_EVENTS table
groups Setting this option will cause group status change events to be written to the AUDIT_EVENTS table
jobs Setting this option will cause job start and stop events to be audited
job-data Setting this option will cause job instance start and stop events to be audited (each time a job is executed an event will be written to AUDIT_EVENTS when the job starts and then when it stops)
oload Setting this option will cause oload to write audit events when source code is loaded into the system schema
services Setting this option will audit service start and stop events
system Setting this option will audit system startup, shutdown, and recovery events
user-events Setting this option will cause every call to audit_user_event() to be written to the AUDIT_EVENTS table; if this audit option is not set, then calls to audit_user_event() will be discarded
workflow-data Setting this option will audit workflow status changes; note that this will have a performance impact as each change to a workflow order's status will cause a row to be written in the AUDIT_EVENTS table
workflows Setting this option will audit workflow start and stop events
Note
If an event is not audited, then it is neither written to the database nor logged in the audit log file.

Audit Events

Audit events are written to the AUDIT_EVENTS table; each audit event has attributes as in the following table:

Audit Event Attributes

Column Type Description
audit_eventid number a unique number assigned to the event (created from a database sequence)
[related_audit_eventid] number an optional event id that this event is related to
[workflowid] number if the event is related to a workflow, the workflowid
[workflow_instanceid] number if the event is related to a workflow instance, the workflow instance id
[stepid] number if the event is related to a workflow step, the stepid
[ind] number if the event is related to a step, the step index number (always 0 for non array steps)
[jobid] number if the event is related to a job, the jobid
[job_instanceid] number if the event is related to a job instance, the job instance id
[serviceid] number if the event is related to a service, the serviceid
audit_event_code number a code giving the audit event type; see Audit Event Codes for possible values
[audit_user_event] string for user events, the user event code
[reason] string a reason for the event (ex: if due to an API call, then the API name, ex "omq.system.shutdown", if due to normal system processing, then the string "internal call")
who string the RBAC user who initiated the action that led to the event, or "Qorus" if due to normal system processing or no username is available
source string the source of the call that led to the event (ex: "source: ipv6[::1]:50653 listener: ipv6[::]:8001") or "system" if due to normal system processing
[info1] string an informational string related to the event
[info2] string an informational string related to the event
created date/time the date and time the event was created in the database

System Events

Qorus includes an event sink that maintains a list of system events. System events can be read internally and externally through the system API.

The system option qorus.max-events controls the maximum number of system events the system will store.

System events are identified by an event class and an event code, and each event also contains addtional information describing details of the event. Note that there is a special system event, USER_EVENT, that is used for all user events; user events are differentiated by their custom content as supplied by the user code when the event is raised.

The following give some links to additional information about events:

Database Transaction Model

State Machine

Basically the Qorus Integration Engine® system can be considered a workflow state machine that uses the Oracle database to store the workflow state data.

After every atomic transaction any changes to the database are immediately committed. By design there are no cases where state information relevant to state recovery after a crash is internally queued in the system and committed later. While this can have a negative performance impact, it reduces the impact of system crashes and makes complete system/workflow recovery possible (as long as the database remains intact).

Note
Never update the Qorus system schema directly using SQL; always system APIs. Any inconsistencies in the system schema can lead to data corruption, deadlocks, or other errors. Additionally any row, table, or other locks created manually when Qorus is running can lead to deadlocks; never lock any database object using SQL in the system schema while the system is running.

Access to the system schema where transaction management is required is made through a connection pool, where connections are automatically assigned to threads as needed. Connections are acquired when the first statements that make changes to the database are executed and released to the pool when each thread does a commit or rollback on the connection.

Note
As documented in the Installation Guide, never use Oracle load sharing with the Qorus system schema. The delay caused by Oracle cluster synchronization will cause problems when one or more threads start transactions in other database connections that depend on the results of another thread that has just committed its transaction. If each connection may be talking to a different physical cluster node, then the results of synchronized transactions are not immediately available to other threads, however for performance reasons Qorus depends on this being the case.

Recovery Model

Due to the system's design, the worst-case scenario for a system crash is that workflow order data instances (i.e. also segment and step instances) will incorrectly have an OMQ::StatInProgress status in the database after the crash. That means that the information loss is well defined and limited to the bottom level of the workflow order data instance; the steps with an OMQ::StatInProgress status may have been subject to information loss.

When a crashed session is recovered, all workflow order data instances, segment instances, and step instances with an OMQ::StatInProgress are modified to OMQ::StatRetry and the workflow order data instance rows (the parent object) are cleared of the crashed session's ID.

A well-defined workflow will have recovery logic where necessary that will verify the status of any errored step before repeating the step's actions (by defining validation logic in the step). In this case, the workflow developer can ensure the automatic recovery of the step with no data loss after a system crash.

HTTP Server

Qorus includes a built-in HTTP server providing a gateway to internal protocol handlers. To better support compression with large messages, the HTTP server understands and can generate "deflate" (RFC1951), "gzip" (RFC1952), and "bzip" content-encodings and will respect client preferences as transmitted in the "Accept-Encodings" HTTP header.

When Role Based Access Control is enabled, every request to the HTTP server must include HTTP authentication; the HTTP username is the Qorus RBAC user, and the password is the user's password (however unauthentication connections can be made from certain hosts if the qorus.rbac-force-user option is set).

A new thread is launched to handle each incoming request so the system can handle many requests simultaneously.

Qorus supports the following top-level paths by default (note that requests are also matched by Content-Type if applicable):

Qorus HTTP Server Protocol Handlers

Path Content-Type Handler Description
/YAML application/x-yaml YamlRpcHandler Handles YAML-RPC requests; acts as a YAML-RPC gateway to the system API
/RPC2 text/xml XmlRpcHandler Handles XML-RPC requests; acts as an XML-RPC gateway to the system API
/JSON application/json JsonRpcHandler Handles JSON-RPC requests; acts as a JSON-RPC gateway to the system API
/SOAP text/xml or application/soap+xml SoapHandler Handles SOAP requests for user services exported with svc_register_soap_handler()
/api application/x-yaml, text/xml, application/json REST Implements the system REST API
/apievents n/a WebSocketHandler Handles requests for system events over the RFC-6455 WebSocket protocol
/log/* n/a WebSocketHandler Handles requests for log file events over the RFC-6455 WebSocket protocol

Any requests not matched with a protocol handler are handled by the default handler, which serves the system HTML5/AJAX web UI.