 |
Qorus Integration Engine® Community Edition 6.0.0_dev
|
|
Qorus Integration Engine® Community Edition 6.0.0_dev
|
Back to the System Reference Manual Table of Contents
Qorus Integration Engine® provides a framework for the reliable operation of business-critical interfaces. From a high level, the Qorus Integration Engine® server is made up of a set of processes backed by metadata and status information stored in a database schema, communicating with other systems and clients through the network or the enterprise message bus as follows:
The following figure gives a high-level overview of the internal system architecture:
The Qorus system provides a workflow logic cache, a workflow API, and a workflow execution engine, a service loader, a service API, a job engine and API, many other APIs, and a multithreaded HTTP server servicing REST, YAML-RPC, XML-RPC, JSON-RPC, WebSocket, and HTTP requests, among others. Because the REST, YAML-RPC, XML-RPC, and JSON-RPC handlers export all service methods automatically as well as built-in system API methods, user services can be considered run-time extensible and run-time upgradeable API providers as well (and can explicitly serve APIs using the API manager configuration of a service, for example).
The APIs made available by the Qorus system are listed in the following table.
Qorus APIs
| API | Type | Description |
| Workflow API | Internal | Set of functionality exported for use by workflow code |
| Service API | Internal | Set of functionality exported for use by services |
| Job API | Internal | Set of functionality for use by job code |
| Mapper API | Internal | Set of functionality for use by mapper code |
| System Service API | Internal and External | Set of services delivered with the system providing additional functionality to Qorus |
| REST API | External | Qorus system network API |
Qorus relies on its system schema to store most of its configuration information, some system logic (in system services), and all user configuration.
The following table outlines some of the data that is stored in the system database.
Qorus Database Configuration
| Data | Description |
| Workflow Metadata | The description of a workflow (name, version, steps, dependencies, etc) and all workflow logic written in Python, Java, or Qore using the workflow API |
| Service Metadata | The description of a system or user service (method names, version, etc) and service method logic written in Python, Java, or Qore and using the service API |
| Job Metadata | The descripion of time-based jobs and job logic using the job API |
| Workflow Order Data | Workflow order data instances, the data and processing status |
Qorus is a highly multithreaded integration platform. Due to the fact that much of the system code is stored in a database, and can be "refreshed" (i.e. old version deleted from the cache, new version read in from the database and started) while the system is running, not only can use logic be upgraded without shutting the system down, parts of the workflow system itself (system services) can also be upgraded without shutting the system down.
The majority of the additional programs included with Qorus are client programs to the integration engine. They communicate with the Qorus server either via the HTTP server using REST, YAML-RPC, XML-RPC, or JSON-RPC protocols or via ZeroMQ for clients using the high-performance encrypted network cluster API.
Qorus features a built-in multithreaded HTTP server which functions as the system's primary interface to the outside world and serves various protocols as well as HTTP/S.
All system activities happen in a set of processes that manage many concurrent threads to get their work done.
For self-contained application installations; the Qorus Integration Engine® application directory has a structure as given in the following table.
Qorus Integration Engine® Self-Contained Application Directory Layout
| Directory | Description |
$OMQ_DIR/bin | System binaries are located here |
$OMQ_DIR/etc | System configuration files |
$OMQ_DIR/jar | Qorus Java files |
$OMQ_DIR/lib | System and included third-party library files |
$OMQ_DIR/qlib | Qore-language include file directory |
$OMQ_DIR/python | Qorus Python modules |
$OMQ_DIR/releases | System load files |
$OMQ_DIR/system | System service definitions |
$OMQ_DIR/templates | Miscellaneous template files |
$OMQ_DIR/user | Qorus user definition files, code objects, interface descriptions, release source |
$OMQ_DIR/webapp | Qorus web UI files |
Contents of this section:
The options file contains system and client options. The location of the file is $OMQ_DIR/etc/options. This file is also mounted on the host system in Docker installations and is located on a shared filesystem in Kubernetes installations.
Qorus System options have the format:
qorus.option: value Example:
qorus.http-server: 8002
Also an equals sign (=) may be used instead of a colon as in:
qorus.http-server = 8002
Any text after a # character is assumed to be a comment.
One of the most important option that should be set before the system will start is instance-key. This must be a unique identifier for the instance that will enable the system to avoid accidentally starting the same instance more than once and will also allow the system to recover the application session gracefully errors should the system ever terminate abnormally. The default value for this key is "qorus-test-instance".
Example qorus.instance-key option setting:
qorus.instance-key: bss-uat-1
Additionally, if the log files for Qorus should be written to a directory other than the default (the default for tar installations is $OMQ_DIR/log), the qorus.logdir option must be set to the directory where Qorus log files will be written. This directory must be writable by the user running the Qorus server:
Example qorus.logdir option setting:
qorus.logdir: /var/log/qorus
Another key of interest in the initial installation is http-secure-server:
Example qorus.http-secure-server option setting:
qorus.http-secure-server: 192.168.20.77:8085{cert=$OMQ_DIR/etc/cert1.pem;key=$OMQ_DIR/etc/key1.pem},8086{cert=$OMQ_DIR/etc/cert2.pem;key=$OMQ_DIR/etc/key2.pem}
qorus.http-secure-server: 8087{cert=$OMQ_DIR/etc/cert3.pem;key=$OMQ_DIR/etc/key3.pem}If no qorus.http-secure-server or qorus.http-server options are set, the default for the HTTP server value is 8001, meaning that Qorus will listen on all interfaces on port 8001 for unencrypted requests, and any APIs requiring encrypted client connections will be unavailable.
Qorus client options have the format:
qorus-client.option: value Example:
qorus-client.proxy-url: http://user:pass@192.168.25.7:8080
As of Qorus 4.0 the dbparams file is no longer used by Qorus; all system datasources are handled as datasource connection objects instead.
When upgrading a per-4.0 version of Qorus, datasources defined in any existing dbparams file are automatically migrated to the system database and created as datasource connection objects.
As of Qorus 4.0 the remoteconnections file is no longer used by Qorus; all Qorus remote connections are handled as Qorus remote connection objects instead.
When upgrading a per-4.0 version of Qorus, Qorus remote connections defined in any existing remoteconnections file are automatically migrated to the system database and created as Qorus remote connection objects.
Once launched, the system initializes itself with the following steps:
Step 1: Parse Options File
Parse System Options File; if any syntax errors are found, information is displayed and the program exits.
Step 2: Open System Datasources
Open System Datasource "omq". If the system "omq" datasource cannot be opened, then error information is displayed and the program exits.
Step 3: Open Log Files
If the log files cannot be opened, an error message is displayed and the program exits.
Step 4: Initialize RBAC Framework
interface groups are read in from the database.
Step 5: Open Application Instance Session
If any errors are encountered, information is logged, and the program exits. The application DB session is recovered here if needed.
Step 6: Initialize Services
Services with the autostart flag set are started automatically. Any errors auto-starting services are logged, but do not otherwise affect system startup.
Step 7: Start HTTP Server
If any errors are encountered, information is logged, any loaded services are deleted, and the program exits.
When the HTTP server is started, listeners for each entry in the System Options File (or as overridden on the command-line) will listen for requests on their respective interfaces and ports. To start and stop workflows and services, to shut down the system, to change options, etc, the appropriate system API calls must be made, normally as properly formatted REST, YAML-RPC, XML-RPC, or JSON-RPC commands sent to Qorus' HTTP server. For more details, please see HTTP Server.
System startup raises the following system event: SYSTEM_STARTUP
If the shutdown message is received, the system performs the following steps:
Step 1: Stop All Workflows
All running workflow execution instances are stopped. While system shutdown is in progress, it is not possible to start new workflow execution instances.
Step 2: Stop all Jobs
All jobs are stopped.
Step 3: Stop and Unload All Services
All system and user services are stopped and unloaded from the service cache.
Step 4: Stop HTTP Server
Any requests in progress must complete before the server can be stopped. After this point, no external communication is possible with the Qorus system.
Step 5: Close Application Session
The application session is marked as closed in the system database.
Step 6: Close System Log Files
System log files are closed.
System shutdown is the last system event raised by the system; the event raised is SYSTEM_SHUTDOWN.
The Community Edition of Qorus does not support users, roles, and permissions; these are only supported in the Enterprise Edition.
RBAC Concepts
| RBAC Concept | Description |
| interface group | An interface group contains lists of workflows, services, jobs, finite state machines, pipelines, mappers, value maps; associated interfaces can be enabled or disabled for operational reasons by enabling or disabling the interface group, and the group can be applied to roles to limit access to those objects through the system API. |
An interface group is contains a list of workflows, user services, jobs, mappers, value maps, finite state machines, and data pipelines; groups can be enabled and disabled.
Interface groups service the following functions:
If a group is disabled, then no workflows, services, or jobs in the group can be started; if a group is disabled while member workflow execution instances, services are running or loaded, or jobs are active, then they will be immediately stopped when the group is disabled.
When an interface group is enabled, then all workflows with a non-zero autostart value are started, all services with their autostart flag set are loaded, and all active jobs in the group are reactivated (assuming they are not also members of another disabled group or they do not have connection dependencies that prohib them to be started).
"DEFAULT" group cannot be disabled or deletedThe "DEFAULT" group (OMQ::DefaultGroupName) with ID 0 (OMQ::DefaultGroupID) is a special system group that automatically contains all system objects.
Qorus supports writing out database records for system and user events; this solution is called system auditing; records are written to the AUDIT_EVENTS table.
System auditing is enabled by setting the qorus.audit system option. Audit events can be used to keep a security record of actions taken in the system, or to explicitly track KPIs (Key Process Indicators) for SLA tracking, for example (see the audit_user_event() function, for example).
The qorus.audit system option accepts the codes in the following table (corresponding to Audit Options).
Audit Option Codes
| Code | Description |
alerts | Setting this option will cause alert events to be written to the AUDIT_EVENTS table |
api | Setting this option will cause every api call that makes changes to be audited in the AUDIT_EVENTS table |
groups | Setting this option will cause group status change events to be written to the AUDIT_EVENTS table |
jobs | Setting this option will cause job start and stop events to be audited |
job-data | Setting this option will cause job instance start and stop events to be audited (each time a job is executed an event will be written to AUDIT_EVENTS when the job starts and then when it stops) |
oload | Setting this option will cause oload to write audit events when source code is loaded into the system schema |
services | Setting this option will audit service start and stop events |
system | Setting this option will audit system startup, shutdown, and recovery events |
user-events | Setting this option will cause every call to audit_user_event() to be written to the AUDIT_EVENTS table; if this audit option is not set, then calls to audit_user_event() will be discarded |
workflow-data | Setting this option will audit workflow status changes; note that this will have a performance impact as each change to a workflow order's status will cause a row to be written in the AUDIT_EVENTS table |
workflows | Setting this option will audit workflow start and stop events |
Audit events are written to the AUDIT_EVENTS table; each audit event has attributes as in the following table:
Audit Event Attributes
| Column | Type | Description |
audit_eventid | number | a unique number assigned to the event (created from a database sequence) |
[related_audit_eventid] | number | an optional event id that this event is related to |
[workflowid] | number | if the event is related to a workflow, the workflowid |
[workflow_instanceid] | number | if the event is related to a workflow instance, the workflow instance id |
[stepid] | number | if the event is related to a workflow step, the stepid |
[ind] | number | if the event is related to a step, the step index number (always 0 for non array steps) |
[jobid] | number | if the event is related to a job, the jobid |
[job_instanceid] | number | if the event is related to a job instance, the job instance id |
[serviceid] | number | if the event is related to a service, the serviceid |
audit_event_code | number | a code giving the audit event type; see Audit Event Codes for possible values |
[audit_user_event] | string | for user events, the user event code |
[reason] | string | a reason for the event (ex: if due to an API call, then the API name, ex "REST PUT api/latest/services/my-example-service/method?action=call", if due to normal system processing, then the string "internal call") |
who | string | the RBAC user who initiated the action that led to the event, or "Qorus" if due to normal system processing or no username is available |
source | string | the source of the call that led to the event (ex: "source: ipv6[::1]:50653 listener: ipv6[::]:8001") or "system" if due to normal system processing |
[info1] | string | an informational string related to the event |
[info2] | string | an informational string related to the event |
created | date/time | the date and time the event was created in the database |
Qorus includes an event sink that maintains a list of system events. System events can be read internally and externally through the system API.
The system option qorus.max-events controls the maximum number of system events the system will store.
System events are identified by an event class and an event code, and each event also contains addtional information describing details of the event. Note that there is a special system event, USER_EVENT, that is used for all user events; user events are differentiated by their custom content as supplied by the user code when the event is raised.
The following give some links to additional information about events:
Basically the Qorus Integration Engine® system can be considered a workflow state machine that uses the Oracle database to store the workflow state data.
After every atomic transaction any changes to the database are immediately committed. By design there are no cases where state information relevant to state recovery after a crash is internally queued in the system and committed later. While this can have a negative performance impact, it reduces the impact of system crashes and makes complete system/workflow recovery possible (as long as the database remains intact).
Access to the system schema where transaction management is required is made through a connection pool, where connections are automatically assigned to threads as needed. Connections are acquired when the first statements that make changes to the database are executed and released to the pool when each thread does a commit or rollback on the connection.
Due to the system's design, the worst-case scenario for a system crash is that workflow order data instances (i.e. also segment and step instances) will incorrectly have an OMQ::StatInProgress status in the database after the crash. That means that the information loss is well defined and limited to the bottom level of the workflow order data instance; the steps with an OMQ::StatInProgress status may have been subject to information loss.
When a crashed session is recovered, all workflow order data instances, segment instances, and step instances with an OMQ::StatInProgress are modified to OMQ::StatRetry and the workflow order data instance rows (the parent object) are cleared of the crashed session's ID.
A well-defined workflow will have recovery logic where necessary that will verify the status of any errored step before repeating the step's actions (by defining validation logic in the step). In this case, the workflow developer can ensure the automatic recovery of the step with no data loss after a system crash.
Qorus includes a built-in HTTP server providing a gateway to internal protocol handlers. To better support compression with large messages, the HTTP server understands and can generate "deflate" (RFC1951), "gzip" (RFC1952), and "bzip" content-encodings and will respect client preferences as transmitted in the "Accept-Encodings" HTTP header.
A new thread is launched to handle each incoming request so the system can handle many requests simultaneously.
Qorus supports the following top-level paths by default (note that requests are also matched by Content-Type if applicable):
Qorus HTTP Server Protocol Handlers
| Path | Content-Type | Handler | Description |
/YAML | application/x-yaml | YamlRpcHandler | Handles YAML-RPC requests; acts as a YAML-RPC gateway to the system API |
/RPC2 | text/xml | XmlRpcHandler | Handles XML-RPC requests; acts as an XML-RPC gateway to the system API |
/JSON | application/json | JsonRpcHandler | Handles JSON-RPC requests; acts as a JSON-RPC gateway to the system API |
/SOAP | text/xml or application/soap+xml | SoapHandler | Handles SOAP requests for user services exported with ServiceApi::registerSoapHandler() |
/api | application/x-yaml, text/xml, application/json | REST | Implements the system REST API |
/apievents | n/a | WebSocketHandler | Handles requests for system events over the RFC-6455 WebSocket protocol |
/creator | n/a | WebSocketHandler | Handles requests for the creator API with the RFC-6455 WebSocket protocol |
/log | n/a | WebSocketHandler | Handles requests for log file events over the RFC-6455 WebSocket protocol |
/remote-command | n/a | WebSocketHandler | Handles requests for remote commands over the RFC-6455 WebSocket protocol |
/webdav | n/a | FsWebDavHandler | Handles requests for WebDAV clients |
Any requests not matched with a protocol handler are handled by the default handler, which serves the system web UI.
@subsesion http_rest_handler REST Handler
The Qorus REST API handler handles requests on the /api URI path.
The following header can be given by clients to specify the client's time zone region:
Qorus-Client-Time-Zone For example:
Qorus-Client-Time-Zone: America/New_York
This allows the REST server to handle date/time values in the client's time zone.
Remote clients can access the $OMQ_DIR/user filesystem using the WebDAV protocol.
Directories with the same name as interface groups that the user has associated with their account are also accessible.