Qorus Integration Engine®  4.0.0_git
Release Notes

Qorus Release Notes History

current: Qorus 4.0.0_git

Release Overview

This is a major release of Qorus with the following major changes:

  • An Application Cluster Implementation
    Separating Qorus interfaces into separate processes brings much improved system reliability and resiliency. Communication between cluster elements is provided by the ZeroMQ library, which provides a high-performance distributed queue layer for reliable information exchange in Qorus. Private network communication is encrypted with an elliptic curve encryption scheme which provides high performance and high security. By isolating interfaces into separate processes and by monitoring and restarting processes when they terminate unexpectedly, the overall stability of the system has been dramatically increased.
  • Support for Reusable Solutions Controlled by Users
    Interface module support allows for new high level integration objects to be defined based on Qorus's fundamental integration objects; these objects can be developed, packaged, and delivered as generic reusable solutions for specific integration challenges or as products on top of the Qorus Integration Engine platform. Interface object configuration item support enables end users to control the runtime behavior of Qorus interfaces and to eliminate the need for development for such changes.
  • Native Java Integration
    Qorus 4.0 introduces complete support for Java, allowing Qorus interface code to be developed directly in Java. See Developing in Java for more information.
  • Updated System Requirements Support for operating systems other than Linux is now only provided through Qorus in Docker containers. See Minimum System Requirements for updated system requirements.
    Note
    Qorus 4.0 will not start interfaces if runtime values of tunable system parameters exceed a threshold of system-defined soft limits; see Minimum System Requirements for more information

Important Upgrade and Backwards Compatibility Information

  • Cluster Functionality
    • Interfaces can now be configured to run in remote mode for overall increased platform reliability at the expense of higher memory usage and a performance penalty regarding intracluster communication. To update interfaces to run internally in qorus-core as with previous versions of Qorus; the following new REST APIs can be called:
    • Running interfaces in remote mode has the following advantages:
      • Bugs in interfaces that could lead to a crash limit the impact of that bug to the interface process and do not affect other interfaces.
      • Memory leaks that cause an interface process to exceed the maximum process memory threshold will cause the errant process to be killed by the Qorus master process, limiting the affect of the memory leak to only the specific interface (see the max-process-memory-percent option).
      • Badly-written interfaces that deadlocks or will not exit or shut down can be killed using the REST API from the command line or the system UI.
      Note that oload will set the remote value to True for any interface that is loaded after the upgrade without an explicit remote option. This behavior can be changed by setting the qorus-client.remote option in the System Options File
    • New programs:
      • qctl: cluster control program
      • qdsp: distributed datasource pool processes
      • qjob: distributed job interface processes
      • qorus-core: central Qorus cluster process handling global services, interface shared state, and non-clustered interfaces
      • qsvc: distributed service interface processes
      • qwf: distributed workflow interface processes
    • Updated programs:
      • qorus: cluster master process; the primary supervisor process for the Qorus cluster on the node where it runs; this process provides the same interface for starting / recovering a Qorus application instance as in previous versions for backwards compatibility
    • New system options:
    • Workflow behavior changes:
      • when resetting a workflow running in a workflow process, the process is stopped and restarted, and after the restart, all running workflow execution instances have new execution IDs
    • Logging updates:
      • log levels are controlled by logger objects and level settings on each logger; the qorus.verbose option has been deprecated and is only used for migrating an indication of the log level to each logger
      • cluster process log files are found in the "cluster" subdirectory of the log directory
      • all stdout and stderr logs from all cluster processes are also redirected to their cluster log files (this includes qorus which no longer prints any output to stdout or stderr while running)
    • Interface definition changes
      • job parameters in job definition files now support the "remote" attribute to define if a job should run in a remote qjob process or not
      • service parameters in service definition files now support the "remote" attribute to define if a service should run in a remote qsvc process or not
      • workflow parameters in workflow definition files now support the "remote" attribute to define if a workflow should run in a remote qwf process or not
    • New system event class and events:
    • System schema table additions:
      • CLASS_DEPENDENCIES
      • CLUSTER_PROCESSES
      • CONNECTION_TYPES
      • GLOBAL_CONFIG_ITEMS
      • JOB_CONFIG_ITEMS
      • JOB_GLOBAL_CONFIG_ITEMS
      • JOB_INSTANCE_STATS
      • JOB_INSTANCE_STATS_STAGE
      • SERVICE_AUTH_LABELS
      • SERVICE_CONFIG_ITEMS
      • SERVICE_GLOBAL_CONFIG_ITEMS
      • STEP_CONFIG_ITEMS
      • WORKFLOW_GLOBAL_CONFIG_ITEMS
    • Only one Qorus instance can be running on a Qorus schema at any one time
  • Class-Based Development From Qorus 4.0 onwards, it is not recommended to use function-based interface development, as all new features and functionality will be implemented in the new class-based APIs; see:
  • Connection Updates
  • Workflow Changes
  • System Option Changes
    • max-events: the default value for this option has changed from 100,000 to 10,000 to reduce memory consumption
    • compat-ignore-empty-order-key: this option will cause Qorus to ignore empty workflow order keys instead of throwing an exception
    • compat-jni-types: this option will disable the more complete and robust automatic data type conversions supported in the updated version of the jni module which was introduced to provide much improved Java integration in Qorus 4.0. This option is recommended to be set for compatibility reasons when upgrading Qorus instances with existing interfaces that make use of the jni module. If this global option is set, it can be disabled in individual interfaces by including %module-cmd(jni) set-compat-types false in Qore code to reenable full automatic type conversions; see JNI Module Compatibility Options for more information.
    • Deprecated the qorus.flush-status option: the only functional effect of enabling this option was a loss of performance, so it has been made non-functional (its value is ignored) and has been deprecated
    • Deprecated the qorus.verbose option: log levels are controlled by logger objects and level settings on each logger; the qorus.verbose option is only used for migrating an indication of the log level to each logger
    • Deprecated the qorus.workflow-params option: this option was complex, generally unused, and better covered by custom error configuration, so it has been made non-functional (its value is ignored) and has been deprecated
    • The info-snapshot option has been removed as workflow order and job result information is always kept up to date in Qorus and the snapshotting mechanism is no longer necessary
    • The dbparams-file option has been removed. DB connections are stored in the system DB.
    • The remoteconnections-file option has been removed. Qorus remote connections are stored in the system DB.
    • The qorus.systemdb option is a new and mandatory option containing a connection string to the system DB. It is migrated automatically from an existing dbparams file when Qorus 4.0 is installed over an existing Qorus installation.
  • System Service Changes
  • REST API v3
    • /api/v3/...: the REST API has been extended and a new version is now available under this URI path prefix; see the previous link for changes in REST API v3
    • the REST API has been updated to return 403 Forbidden responses for authentication and related errors where appropriate
  • Snapshots for workflow and job instances are integral part of Qorus server
    • The info-snapshot mechanism is now fully integrated into the Qorus process and it does not need periodical refreshing. The option is obsolete and has been removed.
  • User Connections
    • All types of connections (user, Qorus remote, and datasource) are stored in the system database. Connections are migrated automatically from existing configuration files when Qorus 4.0 is installed over an existing Qorus installation.
  • RPC API Changes
  • TAR Installation Changes
    • The qore binary delivered with TAR packages no longer sets the module path automatically; to ensure correct operations after upgrading with a TAR package, make sure and set your QORE_MODULE_DIR environment variable accordingly; see Configure Environment Variables for more information

Additional New Features and Changes

Bug Fixes in Qorus 4.0

Issue ID Severity Description
2811 Normal fixed a bug in oload where the active attribute of jobs was not updated when existing jobs were reloaded
2649 Normal fixed a bug in the REST API where it was not possible to create a workflow-specific error without first creating a global error with the same name
2638 Normal fixed a bug in the REST API where arguments given in the URI path were sometimes ignored
2540 Normal fixed a bug where session recovery could deadlock if the connection to the server could not be established and did not time out
2500 Normal the active key of the PUT /api/v3/jobs/{id_or_name}?action=setActive response now reflects the actual job active state
2448 Normal fixed a bug clearing user storage with the REST API
2434 Normal fixed an error in the wf_get_step_info() function documentation regarding the desc key which was previously incorrectly documented to be the description key
2305 Normal workflow errors and warnings with a non-string info argument no longer result in an internal error being raised
2300 Normal fixed a bug where the the error_count and warning_count values were missing on systems using a PostgreSQL DB for the system schema
2225 Normal fixed a bug where the onetimeinit function was executed twice for synchronous workflow orders

Bug Fixes in Qore

See Bug Fixes in Qore

See also