Understanding WSO2 Data Mapper 5.0.0

INTRODUCTION

WSO2 data mapper is a data mapping solution that convert and transform one format of data to a different format. It provides a WSO2 Developer Studio based tool to create the graphical mapping configuration and generate the files needed to execute the mapping configuration by the WSO2 Data mapper engine.

This is the second of WSO2 Data Mapper posts which will explain about WSO2 Data Mapper from the user perspective. First post was to describe the end to end work flow on how to create, deploy and test data mapper configuration.(How to Use WSO2 Data Mapping Mediator in ESB)

COMPONENTS OF WSO2 DATA MAPPER

Before we going into look the operations lets look how things happen under the hood. WSO2 Data Mapper comes as two components ideally. They are Data Mapper Engine and Data Mapper Tooling component.

Data Mapper Tooling component

Data Mapper Tooling component is the interface to user to create configuration files needed for engine to execute the mapping. There are three configuration files needed by the Data Mapper engine. They are,

1. Input schema file
2. Output schema file
3. Mapping configuration file

Input schema and output schema defines the input/output format of input message and required message. It is basically  a custom defined json schema. It will be generated by the Data Mapper tool for you when loading the input and output files.

As above image shows user can load input/output message format in several different ways. They are,

• XML – Sample XML file
• JSON- Sample JSON file
• CSV  – Sample CSV file with column names as the first record
• XSD  – XSD file which is actually schema for XML files
• JSONSCHEMA – WSO2 data mapper json schema
• CONNECTOR

I guess only CONNECTOR option is the confusing part for you. It is to use data mapper with connectors straightly. WSO2 connectors will contain json schemas for each operations which will define the message formats both it will respond and expect. So when user integrate connectors in a project this connector option will search through the workspace and find the available connectors. Then user can select the respective connector and the operation so that related json schema will be loaded for data mapper by the tool.

Mapping configuration file is simply a Java Script file generated looking at the diagram user will draw in the data mapper diagram editor by connecting input elements to output elements. Every operation user defines in diagram will convert to Java Script operation.

So these three files will be generated by the Data Mapper Tool and saved in a Registry Resource project to deploy in a wso2 server.

“.datamapper” and “.datamapper_diagram” files contain meta data related to Data Mapper Diagram. They will be ignored when trying to deploy in a server to use by data mapper engine. Only two schema files and .dmc (data mapper configuration) file will be deploy.

Need an Intermediate Component to Use With WSO2 Products

WSO2 Data Mapper is an independent component. It is not depending on any other WSO2 product. But other products can use data mapper to achieve/offer data mapping capability. For that we need a intermediate component. Data Mapper Mediator is this intermediate component which  will give the data mapping capability into WSO2 ESB.

Data Mapper Mediator will find the configuration files from the registry and configure data mapper engine with the input message type(XML/JSON/CSV) and output message type(XML/JSON/CSV) . Then it will take request message from ESB message flow and use configured data mapper engine to execute the transformation and output message will be added to ESB message flow.

Can Use Product Specific Runtime Variables

Data Mapper engine also given a way to use run time product specific variables in the mapping. The way it works is, intermediate component should construct a map object containing run time product specific variables and send it to  data mapper engine. So when the mapping happens in the data mapper engine, these variables will be available. For Eg: Data Mapper Mediator provides esb axis2/transport/synapse/axis2client/operation/.. properties like this. In the Data Mapper diagram user can use the Property operator and define the scope and the property name and use it in the mapping. Data Mapper Mediator will identify the required properties to execute the mapping and populate a map with the required properties and will send it to engine. We will look it with more detail when we discuss about Property Operator.

Data Mapper Engine

Data Mapper engine need basically following information to be configured,

• Input Message Type
• Output Message Type
• Input schema
• Output schema
• Mapping configuration

At the runtime it will get the input message and runtime variable map object and output the transformed message. To execute the mapping configuration data mapper engine use java scripting api. So if your runtime is JAVA 7 rhino JS engine will be used and if your runtime is JAVA 8 nashorn JS engine will be used.

There are several limitations in the rhino engine that will directly affect data mapper engine when JAVA 7 is used. There are several functions that will not support in rhino,

for eg: String object functions like startsWith() endsWith()

So user may need to aware of that when using custom functions and operators, rhino may have limitations executing those.

OPERATIONS IN WSO2 DATA MAPPER

Let’s look into the operations we could use to do the mapping. Following diagram shows the WSO2 Data Mapping Diagram Editor.

As you can see in the left side we have the operations pallet. These operations can be drag and drop to the Editor area.

There are six categories listed down

• Links
• Common
• Arithmetic
• Conditional
• Boolean
• Type Conversion
• String

LINKS

Data Mapping Link- map elements with other operators and elements.

COMMON

Constant- define string, number or boolean constant values.

Custom Function – define custom function to use in mapping

Properties – use product specific runtime variables

Global Variable – instantiate global variable to access from any where

Compare – compare two inputs in the mapping

ARITHMETIC

Add – add two numbers

Subtract – subtract two or more numbers

Multiply – multiply two or more numbers

Divide –  divide two numbers

Ceiling – get the ceiling value of a number (closest larger integer value)

Floor – get the floor value of a number (closest lower integer value)

Round – get the nearest integer value

Set Precision – Format a number into a specified length

Absolute Value – get the absolute value of a rational number

Min – get the minimum number from given inputs

Max – get the maximum number from given inputs

CONDITIONAL

IfElse – use a condition and select one input from two

BOOLEAN

AND – boolean AND operation on inputs

OR – boolean OR operation on inputs

NOT – boolean NOT operation

TYPE CONVERSION

StringToNumber – convert string value to number (“0” -> 0)

StringToBoolean – convert string value to boolean(“true” -> true)

ToString – convert number or boolean to string

STRING

Concat – concat two or more strings

Split – split a string by a matching string value

Uppercase – convert a string to uppercase letters

Lowercase – convert a string to lowercase letters

String Length – get the length of the string

StratsWith – check whether string starts with a specific value (JAVA7)

EndsWith – check whether string ends with a specific value (JAVA7)

Substring – extract a part of the string value

Trim – remove white spaces from beginning and end of a string

Replace – replace the first occurrence of a target string with other

Match – check whether the input match with a (JS)regular expression

We will discuss more on some of the operations in the next post.

 

• developer studio
• esb
• wso2