Mapper Module

Table of Contents

• Mapper Module Introduction
• Mapper Examples
• Mapper Specification Format
• Mapper Options
• Release Notes
  • Mapper v1.1
  • Mapper v1.0

Mapper Module Introduction

This module provides classes that help with structured data mapping, meaning the transformation of data in one or more input formats to a different output format.

Classes provided by this module:

• Mapper: the base data mapping class
• AbstractMapperIterator: an abstract base class for iterator mapper classes
• MapperIterator: a class that automatically applies a data mapper to iterated data

Mapper Examples

The following is an example map hash with comments:

const DataMap = (
    # output field: "id" mapper from the "Id" element of any "^attributes^" hash in the input record
    "id": "^attributes^.Id",
    # output field: "name": maps from an input field with the same name (no translations are made)
    "name": True,
    # output field: "explicit_count": maps from the input "Count" field, if any value is present then it is converted to an integer
    "explicit_count": ("type": "int", "name": "Count"),
    # output field: "implicit_count": runs the given code on the input record and retuns the result, the code returns the number of "Products" sub-records
    "implicit_count": int sub (any $ignored, hash $rec) { return $rec.Products.size(); },
    # output field: "order_date": converts the "OrderDate" string input field to a date in the specified format
    "order_date": ("name": "OrderDate", "date_format": "DD.MM.YYYY HH:mm:SS.us"),
);

If this map is applied to the following data in the following way:

const MapInput = ((
    "^attributes^": ("Id": 1),
    "name": "John Smith",
    "Count": 1,
    "OrderDate": "02.01.2014 10:37:45.103948",
    "Products": ((
        "ProductName": "Widget 1",
        "Quantity": 1,
        ),
    )), (
    "^attributes^": ("Id": 2),
    "name": "Steve Austin",
    "Count": 2,
    "OrderDate": "04.01.2014 19:21:08.882634",
    "Products": ((
        "ProductName": "Widget X",
        "Quantity": 4,
        ), (
        "ProductName": "Widget 2",
        "Quantity": 2,
        ),
    )),
);

my Mapper $map(DataMap);
my list $l = $map.mapAll(MapInput);
printf("%N\n", $l);

The result will be:

list: (2 elements)
  [0]=hash: (5 members)
    id : 1
    name : "John Smith"
    explicit_count : 1
    implicit_count : 1
    order_date : 2014-01-02 10:37:45.103948 Thu +01:00 (CET)
  [1]=hash: (5 members)
    id : 2
    name : "Steve Austin"
    explicit_count : 2
    implicit_count : 2
    order_date : 2014-01-04 19:21:08.882634 Sat +01:00 (CET))

Mapper Specification Format

The mapper hash is made up of target (ie output) field names (note that dotted output field names result in a nested hash output unless the allow_output_dot option is set) as the key values assigned to field specifications as follows:

• True: this is a shortcut meaning map from an input field with the same name
• a string: giving the input field name directly (equivalent to a hash with the "name" key)
• a closure or call reference: meaning map from a field of the same name an apply the given code to give the value for the mapping (equivalent to a hash with the "code" key); the closure or call reference must accept the following arguments:
  • any $value: the input field value (with the same name as the output field; to use a different name, see the code hash option below)
  • hash $rec: the current input record
• a hash describing the mapping; the following keys are all optional (an empty hash means map from an input field with the same name with no translations):
  • "name": the value of this key gives the name of the input field; only use this if the input record name is different than the output field name; note that if this value contains "." characters and the allow_dot option is not set (see Mapper Options), then the value will be treated like "struct" (the "struct" key value will be created automatically); cannot be used with the "constant" key
  • "struct": the value of this key gives the location of the input field in an input hash in dot notation, ex: "element.name" would look for the field's value in the "name" key of the "element" hash in the input record; cannot be used with the "constant" key
  • "constant": the value of this key will be returned as a constant value; this key cannot be used with the "name", "struct", "code", or "default" keys
  • "code": a closure or call reference to process the field data; cannot be used with the "constant" key
  • "date_format": gives the format for converting a string to a date; see date_formatting for the format of this string; note that this also implies "type" = "date"
  • "default": gives a default value for the field in case no input or translated value is provided
  • "maxlen": an integer giving the maximum output string field length in bytes
  • "number_format": gives the format for converting a string to a number; note that this also implies "type" = "number"
  • "trunc": assign to boolean True if the field should be truncated if over the maximum field length; this key can only be set to True if the "maxlen" key is also given
  • "mand": assign to boolean True if the field is mandatory and an exception should be thrown if no input data is supplied
  • "type": this gives the output field type, can be:
    • "number": field accepts only numeric values (any non-numeric values on input will cause an exception to be thrown when mapping)
    • "int": fields accepts only integer values (any non-integer values on input will cause an exception to be thrown when mapping; note: also "integer" is accepted as an alias for "int")
    • "date": date field

Mapper Options

Mapper objects accept the following options in the option hash:

• "allow_dot": if True (as evaluated by parse_boolean()) then field names with "." characters do not imply a structured internal element lookup; in this case input field names may have "." characters in them, use the "struct" key to use structured internal element loopups (see Mapper Specification Format "struct" docs for more info)
• "allow_output_dot": if True (as evaluated by parse_boolean()) then output field names with "." characters do not imply a structured/hash output element; in this case output field names may have "." characters in them
• "date_format": gives the global format for converting a string to a date; see date_formatting for the format of this string; this is applied to all fields of type "date" unless the field has a "date_format" value that overrides this global setting
• "enc": the output character encoding; if not present then "UTF-8" is assumed
• "input_log": an optional input data logging callback; must accept a hash giving the input data hash
• "input_timezone": an optional string or integer (giving seconds east of UTC) giving the time zone for parsing input data (ex: "Europe/Prague"), if not set defaults to the current TimeZone (see Qore::TimeZone::get())
• "name": the name of the mapper for use in logging and error strings
• "number_format": gives the global format for converting a string to a number; this is applied to all fields of type "number" unless the field has a "number_format" value that overrides this global setting
• "output_log": an optional output data logging callback; must accept a hash giving the output data hash
• "timezone": an optional string or integer (giving seconds east of UTC) giving the time zone definition for output data (ex: "Europe/Prague"), if not set defaults to the current TimeZone (see Qore::TimeZone::get())
• "trunc": an optional closure or call reference for a callback for field truncation warnings; must accept a single string giving the warning text
• "trunc_all": if True (as evaluated by parse_boolean()) then any field without a "trunc" key (see Mapper Specification Format "trunc" description) will automatically be truncated if a "maxlen" attribute is set for the field

Release Notes

Mapper v1.1

• implemented "constant" field tag giving a constant value for the output of a field
• implemented structured output for dotted output field names and the allow_output_dot option to suppress this behavior
• implemented "default" field tag giving a default value if no input value is specified
• moved field length checks after all transformations have been applied
• implemented a global "date_format" mapper option
• implemented the "number_format" field option and a global option of the same name
• fixed bugs in the "timezone" and "input_timezone" options, documented those options

Mapper v1.0

• Initial release