Use Cases for Qorus

Qorus can be used for the following use cases:

Stateful IT process automation: Qorus workflows were designed to handle high-volume IT process automation with the automatic detection and recovery from technical IT errors (ex: database down, API listener temporarily not reachable, network unreliable, etc), and includes APIs for handling sensitive data as well as SLA tracking and reporting and more.
Data integration / ETL challenges: Qorus supports data pipelines, mappers, jobs for scheduled tasks with an audit trail, services for event- and API-driven integration and much more to enable data integration challenges to be solved in less time, with less effort and lower risk than other solutions
CI/CD: Qorus supports CI/CD through automating release packaging and deployments and through facilitating testing to ensure high-quality releases
Data migrations, API handlers, microservices, and more can additionally be handled with ease using Qorus

Overview

This documentation describes how to develop integration as well as AI/ML/Data Science solutions with Qorus Integration Engine™.

Qorus is designed to support no-code/low-code solutions by combining configurable building blocks in end-to-end solutions to solve simple and complex IT integration and automation challenges.

Building blocks are generally defined as class objects, which are a combination of code and configuration that make the code generically useful in as many use cases as possible.

Code in Qorus can be defined in any of the following programming languages:

Python (standard CPython)
Java (OpenJDK JVM)
Qore (designed for Qorus)

The concept of using a dynamically-typed language such as Python in an IT data and process orchestration / automation solution, embedded in a rigid framework to provide interface reliability, error recoverability, and operational traceability such as with Qorus Integration Engine™, results in extremely fast and flexible development and very high operational reliability. This leads directly to lower development and operational costs, greater business flexibility, and higher process quality.

Java is a compiled language bringing enterprise capabilities to Qorus and can be used as the primary language for building-block development, or it can be mixed with Python and Qore code or with no-code building-blocks regardless or the language as needed.

The Qore language is an enabling technology that provides the infrastructure necessary to allow Java and Python to be seamlessly integrated, allowing Python-based AI and data science functionality to interface directly with enterprise systems, data, and protocols through Java APIs.

Qorus code can use APIs in any supported programming language equally; dynamic imports of APIs in other languages are supported in each language; Python can dynamically import Java and Qore APIs, and Java also can dynamically import Python and Qore APIs in the same way using standard import statements in each language.

This low-level language integration is one way that Qorus enables powerful AI/ML/Data Science solutions to easily integrate with enterprise data sources as well as to power fault-tolerant process orchestration.

Another important aspect of Qorus is the Visual-Studio-Code-based IDE, which enables non-expert users to put together and configure end-to-end solutions without coding using building blocks, as well as permitting expert users to build and extend reusable and generic functionality combining code and metadata in building blocks, empowering operations and non-technical users as well as experts to deliver complex end-to-end IT integration and automation solutions on Qorus in less time and with less risk than with alternative solutions.

Programming Languages

Qorus supports Python, Java, and Qore for coding, and each language has its own advantages and disadvantages, however due to the fact that Qore provides the glue that links Python and Java together, all three languages can import and use APIs from the other two.

Qorus Language Table

Attribute	Python	Java	Qore
Threading	Poor	Excellent	Excellent
Memory Footprint	Large	Large	Small
Sandboxing	Poor	Possible	Excellent

Note: Qorus provides complete and transparent API interoperability between the three supported languages; import Java APIs in Python or vice-versa using standard import statements; when importing Qore or Python APIs into Java, wrapper classes for the imported APIs are generated automatically using Qorus's dynamic bytecode generation, for example.

See also

No-Code Platform for Coders

Qorus is a "no-code platform for coders", because it supports coding and also wrapping code into reusable objects driven by configuration as well. This allows developers to prototype solutions and then deliver solutions as reusable, self-documenting components to reduce time, risk, and cost of solving similar problems in the future, as well as reducing the time, risk, and cost of software maintenance of such solutions.

In addition to configuring existing integration objects, all Qorus integration objects backed by code can be implemented, extended, or customized with the Qorus IDE.

No-Code / Low-Code Objects

No-Code / Low-Code Object	Description	Code	Description
Class	Fundamental logic element of Qorus	Yes	Classes allow logic to be encapsulated and made generic by declaring configuration and other attributes that enable classes to be self-documenting, to work in generic contexts, and to be deployed and used by non-expert users
Class Connections	Linear logic flow for interfaces	Yes	Each element in a class connection is either a class connector or a mapper, each of which can be supported by code (in the case of class connectors, there is always code)
Connection	Connection to an external system or data	No	Connections are made up of configuration data and do not contain any code
Connector	Simple linear logic element in a class	Yes	Connectors are special methods in a classes that can be used along with mappers in class connections to be executed as linear logic on a certain event
Data Pipeline	Data processing pipelines	Yes	Data pipeline processors are backed by code at the implementation level; mappers are mostly configuration, but can be customized with code as well
Flows / Finite State Machines	Graphically-created logic flows that can be used in place of code anywhere in Qorus	Yes	Finite state machines are made up of states, some of which (ex: class connector states) are backed by code (classes) at the implementation level
Group	Interface groups	No	Groups are made up of pure configuration
Job	Scheduled tasks with an audit trail	Yes	Jobs are backed by code at the lowest level
Mapper	Data transformation objects	Yes	Mappers are mostly made up of configuration but can be customized with code as well
Queue	Asynchronous queues	No	Queues are made up of pure configuration and data
Service	APIs, protocol handlers, data cache, event listeners, etc	Yes	Services and service methods are backed by code at the implementation level
Step	An atomic logic element in a workflow	Yes	Steps are implemented with classes, finite state machines, or with class connectors, all of which are backed by code at the implementation level
Type	Data type definitions	No	Types describe data types to ensure type-safe data transfers and internal and external communications with Qorus
Value Map	Lookup tables	No	Value maps are made up of pure configuration and data and do not contain any code
Workflow	Stateful IT process orchestration	Yes	Steps and workflow methods are backed by code at the implementation level
Workflow Synchronization Event	Used to synchronize workflow processing	No	Workflow synchronization events are made up of pure configuration and data and do not contain any code

Config Items: Code to No-Code Runtime Configuration for Your Code

The primary way to package code in a reusable, self-documenting object in Qorus is to implement the code in an object that supports config items, therefore config items are one of the main vehicles for enabling developers to package their code as self-documenting no-code objects.

Config items are provided as metadata in supported objects and allow users to control the behavior of the object's code at runtime. Because objects that support config items also support builtin documentation in the form of markdown description fields, this enables objects supporting a combination of code and config items to be reusable and self-documenting.

Here is a view of the IDE where config items can be created or edited:

Creating / Editing Config Items in the IDE

Config items have the properties given in the following table.

Config Item Properties

Property	Type	Description
`allowed_values`	list	A list of discrete allowed values for the config item; if set then the user will have to choose from a value in this list to set the config item's value
`config_group`	`string`	The name of the configuration group; used to separate config items logically in UIs
`default_value`	`any`	Any default value for the config item
`description`	`string`	The markdown description of the config item and how its value affects the behavior of the code at runtime
`name`	`string`	The string value that is used with API calls to use the config item at runtime
`strictly_local`	bool	If `true` then the config item only takes local values and cannot be set with a global config item value
`type`	`string`	The data type of the config item

Then config items can be used at runtime with API calls to UserApi::getConfigItemValue() in the code as in the following examples.

Python Config Item Code Example: def processInputFromConfig(self, input):

input['result'] = self.processInput(input[UserApi.getConfigItemValue("text-analysis-input", {

"input": input,

})])

return input

Java Config Item Code Example: String topic_array[] = (String[])UserApi.getConfigItemValue("kafka-topics");

UserApi.logInfo("subscribing to: %y", (Object)topic_array);

cons.subscribe(Arrays.asList(topic_array));

Qore Config Item Code Example: class SalesforceStreamBase inherits QorusService, private AbstractSalesforceEventStreamer {

constructor() : AbstractSalesforceEventStreamer(ServiceApi::getConfigItemValue("sf-stream-options")) {

}

init() {

setConnectionAndChannels(ServiceApi::getConfigItemValue("sf-stream-connection-name"),

ServiceApi::getConfigItemValue("sf-stream-channel-names"));

initImpl();

start();

}

}

Config item values can be set from the IDE as in the following graphic.

Config item values can also be set from the UI as in the following graphic.

Setting Config Item Values in the UI

Config item values can also be set from the command line - ex:

prompt$ qrest put jobs/my-job/config/my-config-item value=100

The following objects support config items in Qorus:

Flows / Finite State Machines: Build No-Code Logic Graphically

Flows / finite state machines allow developers to implement logic graphically.

This makes the implementation clear and self-documenting, which, combined with a full set of tests for a release, results in reduced time and risk for subsequent changes and inevitable future software maintenance.

Qorus finite state machines allow for building blocks, DataProvider actions, data transformations (with mappers), logical loops and variables, API calls, and more to be implemented with a graphical, self-documenting approach.

Finite state machines can be reused and nested as needed as well, just like real code.

Finite state machines can be used as:

Jobs can use a finite state machine to execute the run action
Services can use a finite state machine to execute any method as well as for handling API endpoints
Steps can use finite state machines to implement step actions such as the primary step logic, the validation logic, array logic (for array steps), and more

Example Flow in the IDE

Note: Finite state machines are executed on the server side from the configuration; no code is generated for finite state machines; this differs from class connections, which are executed based on code generated directly in the IDE.

See also: Finite State Machines / Flow Designer for more information

Class Connections: Build Simple Linear No-Code Actions

Class connections can be used as a simpler alternative to finite state machines. A class connection is a linear set of class connectors and/or mappers, where each element in the connection list is executed after the last, and the output of each element is passed to the next element for processing.

Class connections can be used as:

Jobs can use a class connection to execute the run action
Services can use a class connection to execute any method
Steps can use class connections to implement step actions such as the primary step logic, the validation logic, array logic (for array steps), and more

See the following graphic for an example of class connections in the IDE.

Class Connection List in the IDE

Note: Class connections are executed from code generated in the IDE based on their configuration; this differs from finite state machines, which are executed on the server side directly from their configuration; no code is generated for finite state machines.

High-Level Interface Design

The three main integration objects in Qorus are:

Workflows: for stateful IT process automation, process data (an order) to a COMPLETE status, recover automatically from technical errors
Jobs: tasks run on a specific schedule
Services: APIs, protocol handlers, data caches, event listeners, or miscellaneous functionality not linked to specific order data like a workflow or run on a schedule like a job

Workflow Design

Workflow are all about order data. If you have order data that must be processed, and the order data should complete processing despite technical errors, then a workflow should be used.

Workflow Order Data Status in the UI

Conversely, if you don't have any order data, then as a rule of thumb, you should not create a workflow order until the order has been acquired (with a job or a service). The uniqueness of workflow orders can be guaranteed with unique order keys in the action or API used to create the order.

Qorus workflows automatically track order execution time as a configurable SLA and are very good for tasks that include asynchrononous coordination with internal and external events as well.

Workflows can also handle hierarchical order processing using subworkflow steps, which are also used for logical branching in workflows, since Qorus workflows have otherwise a fixed structure.

Workflows are made up of atomic steps that represent the bulk of the processing logic to be applied to order data; when all steps in a workflow order get a COMPLETE status, the workflow order itself is also COMPLETE.

Steps are defined by subclassing the appropriate step class in any supported language (using dynamic imports in Python and Java, for example) and can be built with a no-code, low-code, or an "only code" approach in Qorus.

Workflows and steps are versioned; new versions of steps and workflows can be deployed to a running system while the workflow itself is running, however workflows and steps can each have multiple versions of the same workflow and step active at the same time, and please note that workflow and step versioning affects processing and specifically recovery processing; see Workflow Error Recoverability and Workflow Upgrades, Bug Fixes, and Recovery Compatibility for more information.

Note

Finite State Machines have a flexible logical structure but are not stateful like workflows; the main advantage of using a Qorus workflow to automate a task is that a properly designed workflow can detect and recover from technical errors automatically, allowing your IT orchestration task to successfully complete potentially complex processing of order data in an unreliable IT integration environment.
SLAs can be tracked anywhere in Qorus also manually using the UserApi::postSlaEventSuccess() and UserApi::postSlaEventError() APIs from any supported language.

See also: Designing and Implementing Workflows

Job Design

Qorus jobs are for tasks that run on a schedule. Each execution of the job results in a record (with a COMPLETE or ERROR status, plus optional free-form data) which is searchable / retrievable from the REST API and visible in the system UI.

Job Result List in the UI

Jobs are suitable for polling tasks and more generally for activity that runs on a schedule.

Jobs are defined by subclassing the QorusJob class in any supported language (using dynamic imports in Python and Java, for example) and can be built with a no-code, low-code, or an "only code" approach in Qorus.

Only one version of a job can be active in Qorus at any other time. A new version of a job can be deployed to a running Qorus instance at any time, however the old version of the job will be replaced by the new version.

See also: Implementing Jobs

Service Design

Qorus services are a powerful and flexible integration object that can perform many tasks; services can implement server-side REST, SOAP, HTTP, WebSocket, stream handlers, or other protocol handlers; they can implement a data cache, event listeners, and more; services are the only Qorus integration object that can start its own threads as well.

Service Info in the UI

Services are defined by subclassing the QorusService class in any supported language (using dynamic imports in Python and Java, for example) and can be built with a no-code, low-code, or an "only code" approach in Qorus.

Only one version of a service can be active in Qorus at any other time. A new version of a service can be deployed to a running Qorus instance at any time, however the old version of the service will be replaced by the new version.

See also: Implementing Services

Table of Contents