Qorus Integration Engine®  5.1.39_git
Implementing Mappers

Mapper Development Overview

See also
Qorus Mappers for more information

All Qorus integration objects are described by YAML metadata produced by our IDE, which is available as a free (as in free of charge and also open source) extension to Microsoft Visual Studio Code, a multi-platform editor from Microsoft. The Qorus extension is called Qorus Developer Tools and can be installed directly from Visual Studio Code.

Mappers are developed in the IDE, loaded into the system schema with oload and participate in releases built with make-release.

Mappers in the Qorus IDE

Each mapper has its own sandboxed Program container that contains the mapping logic and any library objects imported into it. Mapper Program objects have the following sandboxing restrictions set by default:

For mapper library code, the following parse option is also set to restrict code from performing I/O or logging:

Note
The following system options can cause additional options to be set by default:
See also

Note that workflows, services, and jobs must declare mappers to be able to access them. The following api call should be used to acquire a mapper in your interface code:

Mapper Diagram in the Qorus IDE
See also
Mapper Library for more information.

Mapper Files

Mappers are defined as YAML metadata created by the Qorus IDE.

Mappers have the following attributes:

Mapper Name

The name of the mapper; the name and version together are unique identifiers for the mapper and are used to derive the mapperid (which is generated when the mapper is loaded into the system via oload).

Mapper Version

The version string for the mapper; the name and version together are unique identifiers for the mapper and are used to derive the mapperid (which is generated when the mapper is loaded into the system via oload).

Mapper Patch

A string "patch" label which can be used to show that a mapper was updated while not affecting the mapperid.

The patch label is descriptive only and does not affect the mapper's ID.

Mapper Description

A description for the mapper; the description field supports markdown for formatted output in the UI and IDE.

Mapper Options

An optional hash of mapper options as in the following table

Mapper Option Description
input_provider_search The search criteria for the input provider; see the where_cond option of AbstractDataProvider::searchRecords() for more information on this option
input_request The arguments for input providers using the request/response API
input_request_options Any options to input providers using the request/response API
input_response_error a string indicating that input providers using the request/response API should use the given error response message for the record format
input_search_options The search options for the input provider; see the documentation for the input data provider for more information on supported search options
output_nullable Set all output fields as nullable
output_provider_bulk If this option is used with a record-based output provider, then bulk operations are used with the output provider, committing and flushing final data is performed automatically by Qorus
output_provider_passive If this option is set and a record-based or request-reply output provider is set, then nothing will be written to the output provider when mapping; the output provider will only be used to provide type information for the output record
output_provider_upsert Set to True if upsert operations instead of creation APIs should be used with the output provider. If output_provider_bulk is also set, this indicates that the internal AbstractDataProviderBulkOperation object will use upsert operations instead of insert operations
runtime An initial runtime structure for Mapper Runtime Options
trunc_all If True (as evaluated by parse_boolean()) then all fields will be automatically and silently truncated if the output data exceeds the field's length

Mapper Author

The optional author string for the mapper.

Mapper Library

Mappers support library objects that provide additional code for mapper fields using code to define the logic for the output mapping.

See also
Library Objects for more information

Mapper Groups

Mappers can be members of interface groups like other interface objects.

Mapper Context

The mapper context provides information to Qorus about other contextual information that can be used in a mapper such as a workflow's static data type.

Mapper Input

Mappers can declare input types that define the type of input data or an input provider that provides both the input data type and also a source for the data.

Mapper Output

Mappers can declare output types that define the type of output data or an output provider that provides both the output data type and also a source for writing or processing the data.

Mapper Output Field Options

Mapper output fields can be mapped from input fields or they can be derived from multiple fields, other contextual information, from a Python or Qore expression, or by a method call from a mapper code class.0

This section will outline the use of mapper output field options.

IDE Mapper Output Field Options
Note
REST API GET /api/latest/system/default_mapper_keys returns information about mapper output field keys

Mapper Output Field Option: code

Synopsis

code provides a reference to a mapper code method; the given method is called with an argument providing the currently mapped input field (if any) as well as the entire input hash.

In this way any arbitrary logic can be used to populate the output field.

See also
Mapper Output Field Option: context

Mapper Output Field Option: constant

Synopsis

constant: provides a fixed constant value to the output field; the type of the constant must correspond to the output field type.

Complex values must be entered in YAML format; see YAML Formatted Data for more information.

Note
If this mapper output field option is used, no other options can be set

Mapper Output Field Option: context

Synopsis

context: this field holds a string that is evaluated as a template string with UserApi::expandTemplatedValue() to provide the output field value.

The $local:input template (corresponding to the local_context argument in the above method) is assigned to the input hash value, providing an easy way to reference input data values in the context string.

Assume the following input record for the examples following the example data table.

Example Input Data Record

Input Field Input Field Value
id "UfEydGZk2yn5Urk"
first_name "Joline"
last_name "Customer"
type "Residential"

Mapper Output Field Context Option Example Table

Output Field Context Option Value Output Field Value
$local:input.id-$local:input.last_name "UfEydGZk2yn5Urk-Customer"
$qore-expr:{"$local:input.last_name".upr()} "CUSTOMER"
$qore-expr:{"$local:input.type" == "Business" ? 100 : 200} 200
Note
For more complex output field value generation, use a mapper code method
See also

Mapper Output Field Option: default

Synopsis

default: provides a default value to the output field if no mapping is present or the input mapping provides no value; the type of the default value must correspond to the output field type.

Complex values must be entered in YAML format; see YAML Formatted Data for more information.

Mapper Output Field Option: index

Synopsis

index: allows the current row number in the input data to be used as the output field value. Row numbers start with row 0

The value and the value type of this option determine the output value:

  • int values: the given value is added to the row number (where the first row is row 0), and the resulting value is used as the output field value
  • string values: the given string is prepended to the row number (where the first row is row 0), and the resulting string value is used as the output field value
  • other value types: the result of the Qore + operator applied to the row number (where the first row is row 0) and the argument, and the resulting value is used as the output field value

Mapper Output Field Option: name

Synopsis

name: is used to specify a mapping from an input field; this output field is generated automatically by the IDE when an input field is mapped to an output field

Mapper Output Field Option: runtime

Synopsis

runtime: the name of a field in the "runtime" hash where the output field value should be taken from.

Note
use context instead

Mapper Output Field Option: submapper

Synopsis

submapper: the name or identifier of a submapper that will provide a list of values for this field

Note
  • Provided for future use
  • Cannot be used with bulk APIs

Mapper Output Field Option: submapper_auto_input

Synopsis

submapper_auto_input: provides use the input provider from the submapper for the output of the field; requires submapper_options to provide input configuration for the submapper

Note
  • Provided for future use
  • Cannot be used with bulk APIs

Mapper Output Field Option: submapper_options

Synopsis

submapper_options: allows a parent mapper to affect the options of the submapper based on current data; use $local:input.<field> to refer to mapped data in the current parent mapper input row to affect submapper options.

Note
  • Provided for future use
  • Cannot be used with bulk APIs

Mapper Output Field Option: trunc

Synopsis

trunc: a boolean flag that indicates if the input value should be automatically truncated if it is too large to fit in the output field.

If this value is missing or False, a STRING-TOO-LONG exception is raised in such cases.

Mapper Output Field Option: type_options

Synopsis

type_options: a hash of options as supported by the output data type

Mapper Output Field Option: use_input_record

Synopsis

use_input_record: when True, the entire input record will be used as the value of the output field; only compatible with hash output fields.

Note
If this mapper output field option is used, no other options can be set

Mapper Output Field Option: template

Synopsis

template: if True, the value of the input field will be used as a template; i.e. UserApi::expandTemplatedValue() will be called on the input value, and the result will be used as the output field's value.

Mapper Output Field Option: value_lookup

Synopsis

value_lookup: The given value map and value will be looked up and used as the output value; the format of this option is value-map-name.value-map-key.

This option is equivalent to using context = $value-map:{value-map-name.value-map-key}