Qorus Integration Engine® Enterprise Edition 6.1.0_prod
|
This section of the Qorus Integration Engine® manual describes command-line programs delivered with Qorus. Other command-line programs delivered with Qore are not documented here such as:
qdp:
the Qore DataProvider interfaceqdbg
and qdbg-remote
: Qore debugger and remote Qore debugger (see Debugging Remote Interfaces)sqlutil:
the Qore SqlUtil interfacemake-release
[options] [release-label] [files...]The make-release
script should be used to package Qorus user releases for automated installation. This script will package all files to be delivered and created an automated installation package using oload to load all objects into the target Qorus schema.
The script assumes that user files are stored relative to the current directory; use the –U
, -p
, or -P
options to change.
The script creates the release in $HOME/releases
in a subdirectory named qorus-user-
release-label. To change the target directory, use the –r
option (or set the $QORUS_RELEASE_DIR
environment variable before running the script).
This script creates a release directory with the following contents:
Release Directory Contents
Filename | Description |
install.sh | The automated installer script. This script is copied from $OMQ_DIR/templates |
qorus-user- release-label.tar.gz | The compressed tar archive of the user files to be installed |
releases/qorus-user- release-label.qrf | A loader command file is automatically created for all Qorus objects to be loaded in the new schema |
The install.sh
script in this directory can be edited (actions added to the pre_install
and post_install
functions) if necessary (for example, to delete services or delete cached workflow definitions to ensure that new objects are immediately used).
The release can be tar’ed and compressed for distribution by using the -c
option.
make-release
script should always be used to create new releases as future version of Qorus Integration Engine® will likely include release tracking and rollback support, and make-release
will allow these features to be used transparently.Options Supported by make-release
Long Option | Short | Description |
–python-modules=ARG | -a | Package the given directory as a Python module dir to be installed in the target under $OMQ_DIR/user/python/lib/python <version>/site-packages by default |
–python-dest=ARG | -b | Store files packaged with -a in the given directory under $OMQ_DIR/ |
–compress | -c | Creates a ".tar.bz2" archive of the release created |
–refresh | -f | Includes a command in the release file to refresh all loaded objects in the server automatically |
–full-release | -F | verify release completeness, only with -i |
–install | -i | Execute the install.sh immediately to perform a local installation of the release just created (useful for testing) |
–user=ARG | -u | sets the source directory (when not present, assumes ".") |
–user-src=ARG | -U | sets the root source directory for release components; all release componets should be given as relative paths from this directory |
–prefix=ARG | -p | sets user prefix directory for relative paths (implies -U. ) (note: makes a flat release in this dir) |
–add-prefix=ARG | -P | Sets the directory name prefix for the package (ie -Puser = prepend "user" to the target release directory) |
–load-file=ARG | -l | Create only a loader file (do not give a release label with this option) |
–loader-only=ARG | -L | stage system release; create only loader files here |
–user-sql=ARG | -q | Adds an SQL file to execute in the omquser schema when installing |
–release-dir=ARG | -r | Sets the directory where the release will be created (default $QORUS_RELEASE_DIR ) |
–show-release-dir | -R | Show the release directory and exit |
–system-dir=ARG | -s | set system dir (also sets -S ) |
–system | -S | make system release |
–verbose | -v | output more information |
–python-module=ARG | -a | Adds the a python module to the release |
–python-module-dest=ARG | -b | Sets the directory where the python module will be created (default user/python/lib/python <version>/site-packages ) |
–help | -h | Prints command help text |
ocmd
[options] method [arguments]This command provides a command-line interface to the old and deprecated Qorus RPC API. It executes by connecting to the Qorus HTTP server and issuing YAML-RPC, XML-RPC, or JSON-RPC commands. Responses are deserialized into a Qore data structure, and the resulting data is then displayed on the console.
By default, ocmd
uses the YAML-RPC protocol to communicate with the Qorus server.
To submit XML-RPC web service calls, specify xmlrpc
or xmlrpcs
(for XML-RPC over a TLS/SSL-encrypted HTTPS transport) protocol in the URL (ex: "xmlrpc://server1:8501"
), however please note that XML-RPC does not support time zones or fractional second resolution, so date/time information returned by the server may not be displayed correctly by client programs when this protocol is used.
To submit JSON-RPC web service calls, specify jsonrpc
or jsonrpcs
(for JSON-RPC over a TLS/SSL-encrypted HTTPS transport) protocol in the URL (ex: "jsonrpc://server1:8501"
), however please note that JSON does not have native date/time or binary types, so such information returned by the server may not be displayed correctly by client programs when this protocol is used.
Note that command-line arguments are parsed with Util::parse_to_qore_value(), so that complex values can be provided as arguments.
For example, to get help on system API method omq.system.add-user():
Be aware that this program is delivered as a Qore-language script, and, as such, requires either the $OMQ_DIR
or $QORE_MODULE_DIR
environment variables to be set to find the Qorus client module, if not located in the default location.
Options Supported by ocmd
Long Option | Short | Description |
–foreign | -f | Do not process the method name (do not prepend "omq.system." to method names not containing a '.') |
–help | -h | Prints command help text |
–literal | -l | Show literal responses, giving the option 2 times shows HTTP headers |
–monitor | -m | Monitor network events – shows details about all network communication with the Qorus server |
–proxy-url=ARG | -p | Set the proxy URL: "http://host:port" ; at least host or port must be given; ex: "http://proxy1:8080" ; see UNIX Socket Support in URLs |
–remote=ARG | -Q | use the given remote Qorus connection instead of the local instance |
–show-url | -W | Show the system URL as parsed from the option file |
–timeout=ARG | -t | Set the HTTP timeout in seconds (also the connect timeout) |
–unformatted-xml | -X | Shows the literal YAML, XML, or JSON response (unformatted) |
–url=ARG | -u | Set URL: "http://host:port" ; at least host or port must be given; ex: "http://server1:8501/RPC2" ; see UNIX Socket Support in URLs |
–verbose | -v | Shows more information about the web service call |
–xml | -x | Shows the literal YAML, XML, or JSON response (formatted) |
ojview
[options]This program displays information about the status of Qorus Jobs and of job processing. Without options, an overview is given of all job instances in the last 24 hours.
Depending on the options given on the command-line, detailed information can be retrieved and output for the objects desired.
For example, to see an overview of all job data with no time limitations:
unixprompt% ojview –a
To see the detailed status of job instance ID 9649, with job info and error info:
unixprompt% ojview –v j:9649
Be aware that this program is delivered as a Qore-language script, and, as such, requires either the $OMQ_DIR
or $QORE_MODULE_DIR
environment variables to be set to find the Qorus client module, if not located in the default location.
Data Options Supported by ojview
Option | Short | Description |
name | n/a | Shows an overview of the job name given for the given time period (default: last 24 hours) |
number | n/a | Shows an overview of the job id given for the given time period (default: last 24 hours) |
j:id | n/a | Shows information about the job instance given (id = job instance id) |
–all | -a | Show all information |
–date=ARG | -D | Show information from the date and time given (format YYYY[MM[DD[HHmmSS]]] ) |
–day | -d | Show information form the last 24 hours (default) |
–hour | -H | Show information from the last 60 minutes |
–month | -m | Show information from the last month |
–remote=ARG | -Q | use the given remote Qorus connection instead of the local instance |
–week | -w | Show information from the last 7 days |
–year | -y | Show information from the last year |
Metadata Option Supported by ojview
Option | Short | Description |
–definition | -f | Show job definition (metadata) |
–all-jobs | -A | List definition info for all defined jobs |
Output Options Supported by ojview
Option | Short | Description |
–status=ARG | -S | Show only job instances with status=ARG (C = COMPLETE, E = ERROR, I = IN-PROGRESS, Z = CRASH) |
Miscellaneous Options Supported by ojview
Option | Short | Description |
–verbose | -v | Show more information (more –v’s = more info) |
–help | -h | Prints command help text |
–url=ARG | -u | Sets the URL; see UNIX Socket Support in URLs |
–proxy-url=ARG | -p | Set the proxy URL: "http://host:port" ; at least host or port must be given; ex: "http://proxy1:8080" ; see UNIX Socket Support in URLs |
–show-url | -U | Show default URL |
–message | -r | Show reformatted request and response messages |
–literal | -l | Show literal request and response messages (2x or with -v = show headers) |
–mono | -M | Do not use ANSI colors in output |
oload
[options] [workflow|order|service|function file(s) ...]This program is used to load new code and object descriptions into the Qorus database. It can also be used to run SQL files into Qorus datasources, but, as of Qorus version 2.0.0, oload
calls schema-tool to actually execute the SQL. If functions, services, workflows, or orders are changed or inserted, then oload will attempt to refresh the Qorus server’s cache by calling the REST API PUT system/metadata/reload... method. Normally this program should be run by automated installation scripts, however during development this tool may be used to manipulate schema objects by hand.
The loader recognizes the file type based on its extension, therefore metadata files must have the appropriate extensions as documented below.
Metadata File Suffixes
Suffix | Description |
*.qfd | Deprecated Function Definition File |
*.qwf | Deprecated Qore-Language Workflow Definition File |
*.qsd or *.qsd.java | Deprecated Service Definition File |
*.qjob or *.qjob.java | Deprecated Job Definition Files |
*.qclass or *.qclass.java | Deprecated Class Definition File |
*.qconn | Deprecated Connection Definition Files |
*.qconst | Deprecated Constant Definition File |
*.qrf | Loader Release File Description |
*.qsm | User Schema Module Files |
*.qmapper | Deprecated Mapper Files |
*.qscript | Custom Release Script Files |
*.tar* | Qorus release packages with a single subdirectory containing at least install.sh |
*.yaml | YAML Definition Files of Qorus Objects |
If an object already exists, it will be redefined by the oload
program.
The loader supports release files which are useful for packaging automated releases. Release files are text files with simple instructions that are executed by the loader.
Each command takes one or no arguments. The commands in the following table are supported:
Load Release File Commands
Command | Argument | Description |
load | Filename | Loads the specified file into the database |
omquser-exec-sql | Filename | Executes the file as an SQL script using schema-tool in the user schema |
check-shutdown | n/a | Checks if the Qorus server is running; if so, prints an error message and exits |
refresh-all | n/a | Will reset all affected workflows, services, and jobs |
refresh-services | n/a | Will reset all jobs directly loaded in the server |
refresh-recursive | n/a | Will reset all affected workflows, services, and jobs directly listed or used by any library code (functions, classes, constants, etc) loaded |
refresh-services | n/a | Will reset any services directly loaded to ensure that the new configuration is used on the next method call |
refresh-workflows | n/a | Will call omq.system.reset-workflow() to refresh the cached configuration on all workflow updated |
Be aware that this program is delivered as a Qore-language script, and, as such, requires either the $OMQ_DIR
or $QORE_MODULE_DIR
environment variables to be set to find the Qorus client module, if not located in the default location.
Options Supported by oload
Long Option | Short | Description |
–allow-redef | -A | Allows dangerous workflow redefinitions (for example, making recoveries of instances of the previous definition impossible) – for use in development only. |
–data-ts=ARG | Sets the data tablespace name for the given datasource for use with the –schema and –user-schema options, ARG syntax: datasource_name=tablespace_name | |
–datasource=ARG | -D | Overrides datasource parameters, ARG: dsname=spec - spec=driver:user/pass@dbname[(encoding)][hostname][:port] |
–delete=ARG | -X | delete database object, ARG: type:spec - workflow: workflow:name:version - order data: wfinstances:name:version - service: service:name:version - service method: method:servicename:version:methodname - step: step:name:version - function: function:name:version - class: class:name:version - constant: constant:name:version - queue: queue:name
|
–delete-id=ARG | delete database object by id, ARG: type:id - workflow: workflow:id - order data: wfinstances:id - service: service:id - service method: method:id - step: step:id - function: function:id - class: class:id - constant: constant:id - queue: queue:id
| |
–full-release | -F | Error if the release referes to external objects |
–force | -f | Force schema verification/downgrade with user schemas |
–help | -h | Shows option help text |
–index-ts=ARG | Sets the index tablespace name for the given datasource, ARG syntax: datasource_name=tablespace_name | |
–list=ARG | -L | Lists objects in the database, ARG=workflows, services, jobs, classes, constants, functions, queues, events, groups
|
–no-error-update | -E | Do not update workflow error definitions; only create new global error definitions |
–override | -O | Override manually updated entries (job schedule, workflow and service autostart, connection definitions) |
–proxy-url=ARG | -p | Set the proxy URL: "http://host:port" ; at least host or port must be given; ex: "http://proxy1:8080" ; see UNIX Socket Support in URLs |
–quiet | -q | Suppress all output except error messages |
–refresh=ARG | -r | Refreshes objects loaded using the network API, ARG is w (workflows), s (services), or j (jobs) [deprecated by –reload ] |
–reload | -R | Reloads interfaces in the server |
–schema=ARG | Run SQL file given as argument against the omq datasource (only use with system patches) | |
–show-defines | -d | Show defines and exit |
–show-release=ARG | -s | list the specified qorus release content (the argument contains the release name) |
–show-url | -W | Displays the default Qorus system URL |
–sign-schema | Performs no action; schema signing is no longer necessary or supported | |
–token=ARG | -t | Use the given token with HTTP requests to the Qorus server |
–use-schedule | -S | Override manually set job schedules by the schedules from the loaded files (equivalent to qorus-client option override-job-params ) |
–user-schema=ARG | Run SQL file given as argument against the omquser datasource | |
–url=ARG | -u | Set URL: "http://host:port" ; at least host or port must be given; ex: "http://server1:8501" ; see UNIX Socket Support in URLs |
–validate | -l | Validate all workflows, services, and jobs affected by updated or inserted objects; without this option only workflows, services, and jobs that are directly loaded are validated |
–verbose | -v | Sets verbosity level |
The following client options affect the functionality of oload:
-u
)–data-ts
)–index-ts
)-p
)oprop
[options] [[get|set] [domain [key [value]]]]This script provides easy API-based access to Qorus system properties.
Be aware that this program is delivered as a Qore-language script, and, as such, requires either the $OMQ_DIR
or $QORE_MODULE_DIR
environment variables to be set to find the Qorus client module, if not located in the default location.
unixprompt% oprop get [domain [key]]
If only a domain value is supplied, then all properties in the given domain are returned. If a domain and a key value are supplied, then the value of that key in the given domain is returned. If no parameters are suppled, then all system properties in all domains are returned.
To set the value of a system property:
unixprompt% oprop set domain key [value]
If no value is supplied, then the value and the key given are deleted. When the last key of a domain is deleted, the domain entry is removed as well.
Options Supported by oprop
Long Option | Short | Description |
–help | -h | Prints command help text |
–literal | -l | Show literal responses, giving the option 2 times shows HTTP headers |
–proxy-url=ARG | -p | Set the proxy URL: "http://host:port" ; at least host or port must be given; ex: "http://proxy1:8080" ; see UNIX Socket Support in URLs |
–push=ARG | -p | read properties from the given file and publish them to the server |
–reformat | -R | reformat data with -l+ for better readability |
–remote=ARG | -Q | use the given remote Qorus connection instead of the local instance |
–retrieve=ARG | -r | retrieve and save current properties to the given file |
–show-url | -W | Shows the default system URL |
–timeout=ARG | -t | Set the HTTP timeout in seconds (also the connect timeout) |
–url=ARG | -u | Set URL: "http://host:port" ; at least host or port must be given; ex: "http://server1:8501" ; see UNIX Socket Support in URLs |
–verbose | -v | be more verbose |
–xml | -x | use XML instead of YAML (YAML is the default) |
ostart
[options] name [version] [mode] [option=value]This program allows the user to easily start execution instances of a workflow and optionally set options on the instance. If the version argument is not given on the command-line, the latest version of the workflow will be assumed. If the mode value is not given on the command-line, then the mode is assumed to be OMQ::WM_Normal.
For example, to start an execution instance of the latest version of workflow CREATE-ACCOUNT
:
unixprompt% ostart CREATE-ACCOUNT
Be aware that this program is delivered as a Qore-language script, and, as such, requires either the $OMQ_DIR
or $QORE_MODULE_DIR
environment variables to be set to find the Qorus client module, if not located in the default location.
Options Supported by ostart
Long Option | Short | Description |
–file=ARG | -f | Reads in an XML file created by ostatus –r and starts all workflows with the options given in the file. |
–help | -h | Prints command help text |
–proxy-url=ARG | -p | Set the proxy URL: "http://host:port" ; at least host or port must be given; ex: "http://proxy1:8080" ; see UNIX Socket Support in URLs |
–remote=ARG | -Q | use the given remote Qorus connection instead of the local instance |
–show-url | -W | Shows the default URL |
–url=ARG | -u | Set URL: "http://host:port" ; at least host or port must be given; ex: "http://server1:8501" ; see UNIX Socket Support in URLs |
ostatus
[options]This program is capable of displaying system, workflow, and service status. Detailed information on workflows, services, and jobs can be retrieved by using the appropriate command-line arguments, however, by default only a summary line is shown.
For example, to display system information and current system option status:
unixprompt% ostatus -oS
For information about the state of workflows and orders in the database, use the oview command; use the ojview command for job information.
Be aware that this program is delivered as a Qore-language script, and, as such, requires either the $OMQ_DIR
or $QORE_MODULE_DIR
environment variables to be set to find the Qorus client module, if not located in the default location.
Options Supported by ostatus
Long Option | Short | Description |
–callstack [=ARG] | -C | Shows a filtered execution call stack of every active thread in the Qorus server; use -CC to show an unfiltered list (to display also waiting threads) |
–help | -h | Prints command help text |
–jobs | -j | Display active job information (use -v for more information) |
–methods | -m | Display method information when displaying service info (use -v or -vv for more information) |
–options | -o | Show the values of current system options |
–prop=ARG | -p | Retrieves all system properties and saves to an XML file with the given name. |
–proxy-url=ARG | -p | Set the proxy URL: "http://host:port" ; at least host or port must be given; ex: "http://proxy1:8080" ; see UNIX Socket Support in URLs |
–remote=ARG | -Q | use the given remote Qorus connection instead of the local instance |
–restore-autostart=ARG | -E | Restores autostart status from a YAML-encoded file created with -A ; names, versions, and workflowids must match for updates to be made; only differences are updated in the system |
–retrieve=ARG | -r | Retrieves the current running workflow list and all options and saves to an XML file for use with ostart |
–save-autostart=ARG | -A | Saves the autostart status to a YAML-encoded file for later restoration with -E |
–services | -s | Display service information (use –v, -vv, or -m for more information) |
–show-url | -W | Shows the default URL |
–system | -S | Show additional system information |
–timeout=ARG | -t | Set the HTTP timeout in seconds (also the connect timeout) |
–url=ARG | -u | Set URL: "http://host:port" ; at least host or port must be given; ex: "http://server1:8501" ; see UNIX Socket Support in URLs |
–verbose | -v | Show more information (more –v’s = more info) |
–workflows | -w | Display running workflow information (use –v or -vv for more information) |
ostop
[options] id|name [version]This program allows the user to easily stop running execution instances of a workflow by either giving the workflow execution instance id or the name and an optional version.
For example, to stop all execution instances of workflow CREATE-ACCOUNT
:
Be aware that this program is delivered as a Qore-language script, and, as such, requires either the $OMQ_DIR
or $QORE_MODULE_DIR
environment variables to be set to find the Qorus client module, if not located in the default location.
Options Supported by ostop
Long Option | Short | Description |
–all | -a | Stop all running workflows |
–help | -h | Prints command help text |
–proxy-url=ARG | -p | Set the proxy URL: "http://host:port" ; at least host or port must be given; ex: "http://proxy1:8080" ; see UNIX Socket Support in URLs |
–remote=ARG | -Q | use the given remote Qorus connection instead of the local instance |
–show-url | -W | Shows the default URL |
–url=ARG | -u | Set URL: "http://host:port" ; at least host or port must be given; ex: "http://server1:8501" ; see UNIX Socket Support in URLs |
oview
[options]This program displays information about the status of workflow order data instances in the database. Without options, an overview is given of all workflow order data instances in the last 24 hours that do not have a COMPLETE status.
Depending on the options given on the command-line, detailed information can be retrieved and output for the objects desired.
Workflow errors can be automatically queried as well, and workflow orders can be automatically retried based on the search results with -R
.
For example, to see an overview of all workflow data with no time limitations:
unixprompt% oview –ca
To see the detailed status of workflow order data instance ID 419649, with step and segment details:
unixprompt% oview –ssg w:419649
For more information about this service, please see the system.info service.
Be aware that this program is delivered as a Qore-language script, and, as such, requires either the $OMQ_DIR
or $QORE_MODULE_DIR
environment variables to be set to find the Qorus client module, if not located in the default location.
Data Options Supported by oview
Option | Short | Description |
name | n/a | Shows an overview of the workflow name given |
number | n/a | Shows an overview of the workflow id given |
w:id | n/a | Shows information about the workflow order data instance given (id = workflow order data instance id) |
e:id | n/a | Shows information about the order given (id = external order data instance id) |
o:id | n/a | Shows order data information about the workflow order data instance ID given (id = workflow order data instance id) |
k:arg | n/a | Workflow order data instance by key, arg=name:version:key:value |
E:code | n/a | Performs a search for workflow errors with the given error code for error searches |
–date=ARG | -D | Show information from the date and time given (format YYYY[MM[DD[HHmmSS]]] ) |
–hour | -H | Show information from the last 60 minutes |
–day | -d | Show information form the last 24 hours (default) |
–week | -w | Show information from the last 7 days |
–month | -m | Show information from the last month |
–year | -y | Show information from the last year |
–all | -a | Show all information |
–complete | -c | Include workflow order data instances with OMQ::StatComplete status in output |
–nosqlcache | -F | do not use SQL cache (exact values, slow with large databases |
Code Options Supported by oview
Option | Description |
class:name[:version] | Displays the given class code object in source file format |
const:name[:version] | Displays the given constant code object in source file format |
func:name[:version] | Displays the given function code object in source file format |
map:name[:version] | Displays the given mapper in source file format |
meth:service[:version]:name | Displays the given service method in source file format |
svc:name[:version] | Displays the given service and all methods in service source file format |
vmap:name | Displays the given value map in source file format |
job:name | Displays the given job in source file format |
conn:name | Displays the given user connection in source file format |
Output Options Supported by oview
Option | Short | Description |
–status=ARG | -S | Show only workflow order data instances with status=ARG: - A = ASYNC-WAITING - B = BLOCKED - C = COMPLETE - E = ERROR - I = IN-PROGRESS - N = INCOMPLETE - R = RETRY - S = SCHEDULED - V = EVENT-WAITING - W = WAITING - X = CANCELED - Y = READY |
–errors | -e | Show detailed error information for workflow order data instances displayed (-v for even more information) |
–full | -f | Show a full list of individual statuses instead of summarized statuses (default is summarized) |
–limit=ARG | -L | limit number of workflows retrieved and displayed |
–segment | -g | Show segment information |
–steps | -s | Show detailed information about steps executed |
–order | -o | Show detailed order information |
Error Search and Retry Options Supported by oview
Option | Short | Description |
–search-errors | -E | apply all other command-line arguments to error searching only |
–show-all-errors | -A | show all errors including warning and info records |
–business | -b | show only business errors |
–technical | -t | show only technical errors |
–stepid=ARG | n/a | show only errors from the given stepid |
–stepname=ARG | n/a | show only errors from the given step name |
–retry | -R | retry all workflow orders found in error search |
Miscellaneous Options Supported by oview
Option | Short | Description |
–help | -h | Prints command help text |
–literal | -l | Show literal request and response messages (2x or with -v = show headers) |
–message | -r | Show reformatted request and response messages |
–mono | -M | Do not use ANSI colors in output |
–proxy-url=ARG | -p | Set the proxy URL: "http://host:port" ; at least host or port must be given; ex: "http://proxy1:8080" ; see UNIX Socket Support in URLs |
–remote=ARG | -Q | use the given remote Qorus connection instead of the local instance |
–show-url | -U | Show default URL |
–url=ARG | -u | Sets the URL; see UNIX Socket Support in URLs |
–verbose | -v | Show more information (more –v’s = more info) |
qbugreport
qbugreport
creates a bug report file to be sent to Qore Technologies for analysis in case of problems with your Qorus instance; use this program when instructed by Qore Technologies support, and submit the resulting bug report file according to the instructions provided.
Example output:
prompt$ qbugreport gathering information for bug report file: "/tmp/qorus-instance-name-bugreport-20230117T185441.tar.bz2" getting processing info for 18 processes: done getting thread stacks for 18 processes: done staged 87 log files getting debug info "sync-cache" from qorus-core: done getting debug info "sync-summary" from qorus-core: done getting debug info "order-stats" from qorus-core: done getting debug info "order-stats-summary" from qorus-core: done getting debug info "services" from qorus-core: done getting debug info "interface-serialization" from qorus-core: done getting debug info "workflow-control" from qorus-core: done getting debug info "jobs" from qorus-core: done getting debug info "sm-data-cache" from qorus-core: done getting debug info "sm-data-cache-summary" from qorus-core: done getting debug info "sm-segment-cache" from qorus-core: done getting debug info "sm-segment-summary" from qorus-core: done getting debug info "sm-local" from qorus-core: done getting debug info "development" from qorus-core: done creating tar file: "/tmp/qorus-instance-name-bugreport-20230117T185441.tar.bz2": done (56230666 bytes) please send "/tmp/qorus-instance-name-bugreport-20230117T185441.tar.bz2" to bugs@qoretechnologies.com
To run in Docker:
docker exec qorus-container-name bash -l -c qbugreport
To run in Kubernetes:
kubectl exec deploy/qorus-core -- bash -l -c qbugreport
As indicated in the output of the program, the final compressed tar file (which in Docker or Kubernetes will be created in the container indicated in the /tmp
directory) should be sent to bugs@.nosp@m.qore.nosp@m.techn.nosp@m.olog.nosp@m.ies.c.nosp@m.om
qctl
[options] <cmd> [args...]qctl
is the primary command-line program for Qorus cluster control and can be used to start, stop, restart, and get information about the cluster, among other things.
qctl start
qctl stop
qctl restart
qctl ps <proc>
qctl info <proc>
qctl threads <proc>
qctl kill <proc>
qctl help
qdsp
is a cluster server process and should only be started by qorus (which in turn should normally be started by qctl).
This process serves a remote database connection pool to other Qorus cluster client processes.
By default qdsp
processes run in centralized or non-coordinated mode, where any DB action through a Qorus datasource will be executed in the qdsp
process and results are serialized and transmitted back to the client process using Qorus's ZeroMQ-based internal cluster API.
This mode is slower than direct database access for heavy SQL I/O, but provides very efficient support for many small queries with stable, long-running connections to the database server in case there is large overhead establishing new database sessions.
When coordinated mode is enabled, for example by setting the qorus.default-datasource-coordinated option to true
or by setting the Qorus datasource option "coord-mode"
to true
for the datasource, each process requesting a connection from the datasource pool requests permission to open a local connection to the database server from the qdsp
process, and all SQL I/O is performed locally in the cluster process.
This mode is recommended for environments where many short-lived connections are acceptable or desired over a few stable, long-running connections, such as with database servers that establish connections very quickly with low overhead, or with databases that implement connection pooling on the server side (ex: Oracle Database Resident Connection Pooling, for example).
When a datasource is reset, the inforation about the any potential new datasource connection parameters is sent to the qdsp
process.
If the coordinated mode has changed, then regardless of the current mode of the qdsp
process, the qdsp
process will immediately terminate (aborting any transactions currently in process if the old mode was not coordinated) and will be restarted by the master process.
Otherwise, the internal datasource connection parameters are updated, and if the qdsp
process is not in coordinated mode, then a new DatasourcePool object is created that will process any new SQL commands received. Transactions still in progress with the old pool are allowed to complete. Because of this, in the worst case if all connections in the old pool are in transactions and the new pool gets new transaction requests immediately for all available connections, up to n + m connections can be used at one time, where n is the maximum number of connections in the old pool, and m is the maximum number of connections in the new pool.
If the qdsp
process terminates or is killed while Qorus is running, then qorus will restart it, and all cluster processes with active connections will be notified that the server terminated abnormally.
If the qdsp
process was in coordnated mode, then there is no affect on cluster processes, but the total number of connections used may exceed the maximum number of allowed connections while existing transactions for connections authorized before the termination are executed.
If the qdsp
process was in non-coordinated mode, then the transaction state for any active connections will have been lost, and cluster client programs are notified, and each thread blocked on an active transaction will get a DATASOURCEPOOL-PROCESS-ERROR
exception raised.
If the qdsp
process was in coordnated mode when a cluster client process terminates during a transaction, then the qdsp
process rolls back any active transactions for the process and returns the connection to the pool.
If the qdsp
process was in non-coordnated mode when a cluster client process terminates while holding a connection reservation slot, the connection is marked as open and immediately provided to any waiting processes.
qevent
[options]qevent
listens for Qorus system events and outputs them to the terminal. Press any key to stop listening and exit the program.
Event Filtering Options Supported by qevent
Long Option | Short | Description |
–and | -A | use AND logic for event filtering and OR NOT logic for negative filtering (default +OR, -AND NOT) |
–class=arg | -c | show event class=arg (initial character match) |
–event=arg | -e | show event=arg (regex match) |
–severity=arg | -y | show events with severity ARG |
–user-severity=arg | -t | show events with user severity ARG |
–user-event=arg | -n | show user events with name ARG |
–not-class=arg | -C | include class ARG in negative filter |
–not-event=arg | -E | include event ARG in negative filter |
–not-severity=arg | -Y | include severity ARG in negative filter |
–not-user-severity=arg | -T | include user severity ARG in negative filter |
–not-user-event=arg | -N | include user event ARG in negative filter |
–raw | -R | show raw event info |
Reference Info Options Supported by qevent
Long Option | Short | Description |
–list-classes | -l | list all class codes and exit |
–list-events | -x | list all event codes and exit |
–list-severity | -z | list all severity levels and exit |
Websocket Options
–list-logs | -L | list online websocket log sources |
–stream-log=ARG | -S | stream given log websocket source |
–jstream=ARG | -J | stream given JSON websocket source |
Miscellaneous Options Supported by qevent
Long Option | Short | Description |
–help | -h | Prints command help text |
–history=ARG | -H | Sets the number of old messages to display on start |
–proxy-url=ARG | -p | Set the proxy URL: "http://host:port" ; at least host or port must be given; ex: "http://proxy1:8080" ; see UNIX Socket Support in URLs |
–remote=ARG | -Q | use the given remote Qorus connection instead of the local instance |
–show-url | -W | Shows the default URL |
–url=ARG | -u | Set URL: "http://host:port" ; at least host or port must be given; ex: "http://server1:8501" ; see UNIX Socket Support in URLs |
–verbose | -v | sets verbosity level (more -v's = more info) |
qhaorder
[options]The qhaorder
program acts as an order proxy server for Qorus; when the Qorus server process is down, qhaorder
will accept and process omq.system.submit-data.[workflow].[version]() API calls directly; creating the order information in the database and returning a correctly-formatted answer. If the Qorus server process is up and running, then qhaorder
will pass through all RPC API calls directly to the Qorus server and return the answer.
qhaorder
starts up an HTTP server with XML-RPC, JSON-RPC, and YAML-RPC protocol handlers (but not REST handlers), like the Qorus HTTP server. It is configured by setting the following options in the options file under the qhaorder
domain:
qhaorder.listeners:
one or more port numbers or interface:port number combinations, for example:qhaorder.listeners: 9001,192.168.20.5:9009The default is to listen on port 9001 on all interfaces
qhaorder.daemon-mode:
True
or False
if qhaorder
should start up and immediately fork in the background, for example:qhaorder.daemon-mode: trueThe default is
True
qhaorder.verbose:
the log level for the proxy server; messages with a log level equal to or lower than this value will be logged, all other messages will be discarded, for example:qhaorder.verbose: 3the default is 2 (equivalent to OMQ::LL_DETAIL_1)
qhaorder
Options Supported by qhaorder
Long Option | Short | Description |
–log-level=ARG | -l | Sets the log level (more -l's = more info) |
–url=ARG | -u | Set URL: "http://host:port" ; at least host or port must be given; ex: "http://server1:8501/RPC2" ; see UNIX Socket Support in URLs |
–proxy-url=ARG | -p | Set the proxy URL: "http://host:port" ; at least host or port must be given; ex: "http://proxy1:8080" ; see UNIX Socket Support in URLs |
–show-url | -W | Shows the default URL |
–help | -h | Prints command help text |
qjob
is a cluster server process and should only be started by qorus (which in turn should normally be started by qctl).
This process is spawned once for each clustered job interface that runs in its own process space.
If the qjob
process terminates or is killed while Qorus is running, then qorus will restart it. After restarting, the qjob
process will recover its application session data in the system schema and restart normal job processing by waiting until the next trigger time.
qjob
processes can be killed with qctl kill <qorus-process-id>
or with the REST API (POST /api/latest/jobs/{id_or_name}?action=kill or POST /api/latest/system/processes/{id}?action=kill) or by killing the operating system process, however interace processes should normally be stopped by disabling the interface; killing an interface process should only be done as a last resort.qorus
[command-line-options] [option=value]This is the Qorus Integration Engine® cluster master or "supervisor" process. It starts, stops, and restarts all other cluster processes on the node where it is running.
For backwards compatibility, this program can also be used to start or recover the Qorus cluster as with previous versions of Qorus, however qctl is recommended for starting (and stopping) the Qorus cluster.
By default this program will launch in the background, however this can be changed by setting the qorus.daemon-mode parameter to False.
prompt$ qorus daemon-mode=0 verbose=6
Command-line Options Supported by qorus
Long Option | Short | Description |
–qorus-dir | -d | Sets the application root directory (overrides $OMQ_DIR) |
–option-list | -l | Shows the list of valid system options and exits |
–help | -h | Shows option help text |
–verbose | -v | Sets verbosity level |
–version | -V | Shows version information and exits |
If the qorus
process terminates or is killed, then it must be restarted for Qorus operation to continue, ottherwise requests to start and stop new interfaces will deadlock due to the missing qorus
master process.
It's recommended to configure the Unix init process to ensure that the qorus
master process is always running, for example by using a systemd
service script for Unix systems that use systemd
for init.
qorus-core
is a cluster server process and should only be started by qorus (which in turn should normally be started by qctl).
This process runs all global services (such as the global HTTP/S listeners) as well as running directly all non-clustered interfaces within its own process space.
If the qorus-core
process terminates or is killed while Qorus is running, then qorus will restart it, and all cluster processes with active connections will be notified that the server terminated abnormally.
The application session will be recovered, and also a recovery for all still-running interface (i.e. qwf, qsvc, and qjob) and qdsp processes is performed.
qrest
[options] [get|put|delete|post] path
[?action=...;arg1=...]This script provides a command-line interface to the Qorus REST API. It sends REST requests to the server according to the command-line options and displays the response. Responses are deserialized into a Qore data structure, and the resulting data is then displayed on the screen. If the HTTP verb is omitted, then GET
is assumed.
The URI path component should not contain the initial /api
component; qrest
prepends /api/latest
to every request; see the following example which calls GET /api/latest/workflows :
By default, qrest
uses YAML serialization for message bodies to communicate with the Qorus server.
Note that message body arguments are parsed with Util::parse_to_qore_value(), so that complex values can be provided as arguments.
Be aware that this program is delivered as a Qore-language script, and, as such, requires the $OMQ_DIR
environment variable to be set to find the Qorus client module.
Options Supported by qrest
Long Option | Short | Description |
–help | -h | Prints command help text |
–literal | -l | Show literal responses, giving the option 2 times shows HTTP headers |
–path=ARG | -P | use alternate leading REST path in URL |
–proxy-url=ARG | -p | Set the proxy URL: "http://host:port" ; at least host or port must be given; ex: "http://proxy1:8080" ; see UNIX Socket Support in URLs |
–reformat | -R | reformat data with -l+ for better readability |
–remote=ARG | -Q | use the given remote Qorus connection instead of the local instance |
–show-url | -W | show default Qorus system REST API URL (with path prefix) and exit |
–timeout=ARG | -t | Set the HTTP timeout in seconds (also the connect timeout) |
qsvc
is a cluster server process and should only be started by qorus (which in turn should normally be started by qctl).
This process is spawned once for each clustered service interface that runs in its own process space.
If the qsvc
process terminates or is killed while Qorus is running, then qorus will restart it if the service's autostart attribute is set to true
.
Stateless services can run in an unlimited number of processes / containers and support a microservice architecture with Qorus.
qsvc
processes can be killed with qctl kill <qorus-process-id>
or with the REST API (POST /api/latest/services/{id_or_name}?action=kill or POST /api/latest/system/processes/{id}?action=kill) or by killing the operating system process, however interace processes should normally be stopped by disabling the interface; killing an interface process should only be done as a last resort.qwf
is a cluster server process and should only be started by qorus (which in turn should normally be started by qctl).
This process is spawned once for each clustered workflow interface that runs in its own process space.
If the qwf
process terminates or is killed while Qorus is running, then qorus will restart it. After restarting, the qwf
process will recover its application session data in the system schema. If the autostart value for the workflow is positive, then workflow processing will restart as well. If not, then the restarted qwf
process will terminate after recovering its application session.
qwf
processes can be killed with qctl kill <qorus-process-id>
or with the REST API (POST /api/latest/workflows/{id_or_name}?action=kill or POST /api/latest/system/processes/{id}?action=kill) or by killing the operating system process, however interace processes should normally be stopped by disabling the interface; killing an interface process should only be done as a last resort.schema-tool
[options]This program assists with schema manipulation, particularly with tasks regarding installing or upgrading the system schema. With no command-line options, the program returns the status of the system schema (according to the configuration found in the Qorus installation pointed to by the $OMQ_DIR
environment variable).
Be aware that this program is delivered as a Qore-language script, and, as such, requires either the $OMQ_DIR
or $QORE_MODULE_DIR
environment variables to be set to find the Qorus client module, if not located in the default location.
schema-tool
supports options for some common database maintenance tasks, as well as takes responsibility for loading all SQL scripts into Qorus datasources. See the options below for more information.
Miscellaneous Options Supported by schema-tool
Long Option | Short | Description |
–code | -c | Display a numeric code instead of the schema version string |
–type | -t | Display only the database driver for the system schema |
–verbose | -v | Show more information when displaying the schema version; additionally displays the archive schema status (if any) |
–arch[=ARC] | -a | Perform the operation on the archive schema; the argument is required with -C |
–datasource=ARG | -D | Overrides datasource parameters, ARG: dsname=spec, spec=driver:user/pass@dbname[hostname][:port] |
–current | Shows the current system (target) schema version | |
–dry-run | -d | do not send any SQL to the database, just prints out commands; affects only 'write' statements |
–migrate | -M | migrate old users in options file to RBAC DB |
–first-time-install | -m | run first time install setup to setup DB archiving and log rotation |
–help | -h | Display this help text |
Schema Maintenance Options Supported by schema-tool
Long Option | Short | Description |
–analyze[=ARG] | -A | Analyze Oracle index health in the given datasource (system schema if no datasource argument is given); verbose info with -v , see also –max-height and –max-leaf-pct |
–block-buffer[=ARG] | -b | Show the Oracle block buffer hit ratio for the given datasource (system schema if no datasource argument is given) |
–force-rebuild[=ARG] | -F | Force rebuilding all indexes in the given datasource (system schema if no datasource argument is given) |
–ignore-errors | -i | ignore errors when loading schema files |
–max-height=ARG | Sets the maximum index height threshold for rebuilding Oracle indexes | |
–max-leaf-pct=ARG | Sets the threshold for the maximum percentage of deleted leaf nodes for rebuilding Oracle indexes | |
–parallel | -P | use "parallel" when rebuilding indexes on Oracle |
–rebuild[=ARG] | -R | Analyze Oracle index health in the given datasource (system schema if no datasource argument is given) and rebuild indexes that exceed the height and deleted leaf node parameters; verbose info with -v , see also –max-height and –max-leaf-pct to set parameters |
–schema-stats[=ARG] | -S | Analyze statistics for all tables in the given datasource (system schema if no datasource argument is given) |
–snapshot | -s | refresh the workflow instances/statuses snapshot |
–table-stats=ARG | -T | Analyze statistics the given table, a datasource may be given if ARG=datasource_name=table_name, otherwise the system schema is assumed if no datasource name is supplied in the argument |
Schema Modification Options Supported by schema-tool
Long Option | Short | Description |
–align-schema | -V | Check the Qorus installation and align the system datasource with the system schema description |
–create | -C | [DEPRECATED] Same as –align |
–upgrade | -U | [DEPRECATED] Same as –align |
–data-ts=ARG | Sets the data tablespace name for the given datasource, ARG syntax: datasource_name=tablespace_name | |
–drop-schema=[ARG] | drop system schema; optional ARG=user schema module for user schema | |
–force | -f | force schema verification and alignment (also allows schema downgrades) |
–ignore-errors | -i | ignore errors when loading schema files |
–index-ts=ARG | Sets the index tablespace name for the given datasource, ARG syntax: datasource_name=tablespace_name | |
–new-release | run release load file for new releases | |
–run-schema=ARG | -r | executes the given schema file in the given datasource, with an optional starting line number for use when recovering after an error: ARG=dsname=filename[:starting-line] |
Session Options Supported by schema-tool
Long Option | Short | Description |
–list-active | -L | list ACTIVE sessions and API reachability status |
–list-sessions[=ARG] | -l | list last ARG sessions (default = 10) |
–session=ARG | -N | show info for given session ID |
–recover[=ARG] | recover application session, ARG=instance name |
swagger2service Petstore.swagger.yaml
swagger2service
[options] url_to_swagger_schemaThis program generates template source code for a server-side REST service for Qorus from a Swagger schema. The service generates two files: a YAML mtadata file and the service source file. The Swagger schema is also referenced as a service resource.
User Options Supported by swagger2service
Long Option | Short | Description |
–building-block | -b | Use the BBM_RestServiceBase class as the base class for the service |
–file=ARG | -f | Specifies the output file name, if not given, then the output files are written to a name derived from the swagger schema |
–list | -L | Lists Swagger schema information such as the target URL, available URI paths and supported HTTP methods; no source output is generated with this option |
–name=ARG | -n | Overrides the output service name; if this option is not supplied, then the service name is derived from the Swagger schema info |
–version=ARG | -v | Overrides the output service version; if this option is not supplied, then the service version is derived from the Swagger schema info |
–help | -h | Display this help text |
user-tool
[options]This program manipulates data in Qorus’ RBAC tables. Users, permissions, role, and access groups can be created, modified, and deleted and information about all RBAC data can be queried.
This program accesses the Qorus database directly, therefore for all options except –synchronize
, it is not subject to Qorus RBAC control itself and does not require the Qorus server to be running. Unless the –synchronize
option is used, only the database server must be available.
user-tool
–s
option to call the PUT /api/latest/system?action=reloadRbac REST API method after effecting updates.Note that arguments below should not contain any spaces unless they are quoted; for example:
unixprompt% user-tool –duser1:name="John Smith"
If there are no spaces in an argument, then quotes are optional.
Be aware that this program is delivered as a Qore-language script, and, as such, requires either the $OMQ_DIR
or $QORE_MODULE_DIR
environment variables to be set to find the Qorus client module, if not located in the default location.
User Options Supported by user-tool
Long Option | Short | Description |
–add-user | -A | Adds a new user; ARG=username,password,role,"description" |
–update-user | -d | Updates an existing user; ARG=user:key=value, valid keys are "pass" , "name" , and "roles" (note that "name" gives the user's descriptive name; currently usernames cannot be changed); roles can be relative, ex: +name,-name |
–add-user-role | -m | Adds one or more roles to an existing user, ARG=user:role[,role...] |
–del-user-role | -O | Removes one or more roles to an existing user, ARG=user:role[,role...] |
–user-info | -i | Displays user information; ARG=user |
–list-users | -l | Displays information about all users; two "l"s lists all roles associated with the users as well (i.e.: user-tool –ll) |
–users-with-role | -w | Display all users with the given role; ARG=role |
Permission Options Supported by user-tool
Long Option | Short | Description |
–add-perm | -P | Adds a new user permission; ARG=name:"description" |
–update-perm | -E | Updates the description for an existing user permission; ARG=name:"description"; only user permissions may be updated |
–list-roles | -L | Lists all user and system permissions |
Role Options Supported by user-tool
Long Option | Short | Description |
–add-role | -R | Adds a new role ARG=role:"description":permission[,...][:group,...] |
–update-role | -U | Updates an existing role; ARG=role:key=value, valid keys are "desc" , "groups" for groups the role is a member of, "perms" for the list of permissions the role has; "groups" and "perms" can be relative; ex: +name,-name |
–add-role-perm | -F | Adds one or more permission to an existing role, ARG=role:permission[,permission...] |
–del-role-perm | -I | Removes one or more roles to an existing role, ARG=role:permission[,permission...] |
–role-info | -f | Displays information about the given role; ARG=role |
–list-roles | -r | Lists all available roles, two "r"s lists all permissions and groups associated with the roles as well (i.e.: user-tool -rr ) |
–role-with-perm | -p | Display all roles with the given permission; ARG=permission |
–add-role-group=ARG | -G | adds one or more groups to a role; ARG=role:group[,group,...] |
–del-role-group=ARG | -o | removes one or more groups from a role; ARG=role:group[,group,...] |
Group Options Supported by user-tool
Long Option | Short | Description |
–add-group=ARG | -g | adds a workflow/service group, arg format: name:key1=value1,key2=value2,... valid keys: "desc" , "workflows" , "services" , and "enabled" . "desc" is required, the "workflows" and "services" keys take comma-separated lists of strings as arguments; if "enabled" is not present, the default is true |
–update-group=ARG | -t | updates a worklfow/service group, arg format: name:key1=value1,key2=value2,... valid keys: "desc" , "workflows" , "services" , "jobs" , and "enabled" . "desc" is required, the "workflows" , "services" , and "jobs" keys take comma-separated lists of strings as arguments and can be relative; ex: +name,-name |
–group-info=ARG | -n | show information on the given group |
–list-groups | -S | lists all groups (two S's gives more info) |
–add-group-wf=ARG | -b | adds a workflow to a workflow/service group; arg format: name=wfname:version,[wfname:version,...] |
–del-group-wf=ARG | -z | deletes a workflow from a workflow/service group; arg format: name=wfname:version,[wfname:version,...] |
–add-group-svc=ARG | -j | adds a service to a workflow/service group; arg format: name=svcname:version,[svcname:version,...] |
–del-group-svc=ARG | -x | deletes a service from a workflow/service group; arg format: name=svcname:version,[svcname:version,...] |
Deletion Options Supported by user-tool
Long Option | Short | Description |
–delete | -X | Deletes an object from the RBAC database; ARG=object:name; valid objects are "user" , "role" , "perm" , and "group" .
|
–cascade-delete | -c | When deleting a permission or group associated to one or more roles, using this option will delete the reference from all roles as well as deleting the permission or group |
Encryption Options Supported by user-tool
Long Option | Short | Description |
–generate-keys | -K | generates encryption keys and system options for sensitive data processing and validates existing encryption options; does not modify existing keys |
Other Options Supported by user-tool
Long Option | Short | Description |
–help | -h | Display this help text |
–datasource | -D | Override the "omq" datasource connection parameters; ex: -Domq=oracle:user/pass@dbname |
–migrate | -M | Migrate all users from $OMQ_DIR/etc/options file (should only be used by the install script) |
–proxy-url=ARG | -p | Set the proxy URL: "http://host:port" ; at least host or port must be given; ex: "http://proxy1:8080" ; see UNIX Socket Support in URLs |
–quiet | -q | Do not display any output unless there is an error |
–rbac-provider | -B | Sets the name of the external RBAC provider to use (ex: "QorusLdapAuth" ) |
–remote=ARG | -Q | use the given remote Qorus connection instead of the local instance |
–show-url | -W | Displays the default Qorus server URL |
–synchronize | -s | Synchronize RBAC database with Qorus server by calling system REST API method PUT /api/latest/system?action=reloadRbac
|
–url=ARG | -u | Sets the Qorus server URL in case the synchronize option is used; see UNIX Socket Support in URLs |