Tag: esb

How to Use WSO2 Data Mapping Mediator in ESB

How to Use WSO2 Data Mapping Mediator in ESB

Please find the updated post from here

https://nuwanpallewela.wordpress.com/2016/07/16/how-to-use-wso2-data-mapping-mediator-in-esb-updated/

INTRODUCTION

WSO2 data mapper is a data mapping solution that convert and transform one format of data to a different format. It provides a WSO2 Developer Studio based tool to create the graphical mapping configuration and generate the files needed to execute the mapping configuration by the WSO2 Data mapper engine as shown in the below diagrams. As the this is really useful feature in message mediation, WSO2 ESB comes with a data mapper mediator which can be integrate into a mediation sequence since WSO2 ESB 5.0.0.

PREREQUISITES

First we need to have the data mapper mediator supported ESB distribution. WSO2 ESB 5.0.0 is the distribution packed with data mapper mediator. You can download alpha pack from here.

And also we need to have a data mapping editor supported Developer Studio ESB Tool. You can follow this article to install the WSO2 Developer Studio ESB Tool 5.0.0.

OVERVIEW

This post will focus on how to create a configuration with Data Mapper mediator. So I will chose a very simple ESB configuration with only a data mapper mediator and respond mediator to check the converted message.

 

employeeToEngineerUserCase.png

Our Employee message will be a xml message and we will send a JSON response to the client. We will use the Postman as out client to send the requests.

Creating Data Mapping Configuration

Lets create a ESB Configuration project named “DataMapperEmployeeToEngineerConfig” from the WSO2 Developer Studio ESB Tool and create a Rest API as follows.

RestAPiWizard.png

Then we can add the Data Mapper Mediator in to the API resource and a Respond mediator to send the converted message back to client.

apiconfig.png

Then double click on the data mapper mediator to configure the mapping. It will open a dialog box asking to create a Registry Resource project. This is to save and deploy the required configuration and schema files in the ESB Server to read at the runtime when message conversion happens. So name the configuration file as “EmployeeToEngineerMapping” and registry resource project as “EmployeeToEngineerRegistryResource”.registryresourcefile.pngIt will open the Data Mapper Diagram Editor where we should do the mapping.

diagrameditor.png

But first we need to complete the API configuration. So go back to ESB Diagram editor with the API Resource and click on the Data Mapper mediator. You will see the configurations of the Data Mapper mediator on the property table.

data mapper configuration.png

As you can see there are five configurations listed. They are

  • Input Type : Expected input message type (xml/json/csv)
  • Output Type: Target Output message type (xml/json/csv)
  • Configuration : This file contain the script file used to execute the mapping
  • Input Schema: JSON schema which represent input message format
  • Output Schema: JSON schema which represent the output message format

As we are going to convert xml message to json message change the output type to json.

propertytable.png

And then click on the API resource and  go to the property table. And change the configurations to following.

  • Url Style : URL_MAPPING
  • URL-Mapping : /employeetoengineer
  • Methods : POST

apiconfigchanges.png

Save the API configuration. Now lets move to data mapper editor to create the mapping from employee message to engineer.

First we need to load the input and output message formats into editor. For the right click on the input box title bar and select load input schema command.

loadschemafromfile.png

Then following dialog box appears and you will have multiple ways to load the input message format.

loadinputclose.png

You can select the resource type as XML, JSON, XSD or JSONSCHEMA.

If you selects XML, you can load a sample xml message you expect and WSO2 Data Mapper Editor will generate the JSON schema to represent the xml accoring to the WSO2 Data Mapper Schema specification.

Same as for XML, if you select  JSON as resource type you can load a sample JSON message.

If you have a xsd schema which defines you xml message format, you can select the resource type as XSD and load the xsd file.

And also if you have the JSON schema for your message according to the WSO2 Data Mapper schema specification you can load it by selecting resource type as JSON schema.

Lets use a sample xml message to load input format to the data mapper editor. Use following xml as the sample input.

<?xml version='1.0' encoding='utf-8'?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Body><ns1:employees xmlns:ns1="http://wso2.employee.info" xmlns:ns2="http://wso2.employee.address">
<ns1:employee>
<ns1:firstname>Mike</ns1:firstname>
<ns1:lastname>Jhonson</ns1:lastname>
<ns2:addresses>
<ns2:address location="home" >
<ns2:city postalcode="30000">KS</ns2:city>
<ns2:road>main rd</ns2:road>
</ns2:address>
<ns2:address location="office" >
<ns2:city postalcode="10003">NY</ns2:city>
<ns2:road>cross street</ns2:road>
</ns2:address>
</ns2:addresses>
</ns1:employee>
<ns1:employee>
<ns1:firstname>Patric</ns1:firstname>
<ns1:lastname>Jane</ns1:lastname>
<ns2:addresses>
<ns2:address location="home" >
<ns2:city postalcode="60000" >Melborne</ns2:city>
<ns2:road>park street</ns2:road>
</ns2:address>
<ns2:address location="office" >
<ns2:city postalcode="10003">NY</ns2:city>
<ns2:road>cross street</ns2:road>
</ns2:address>
</ns2:addresses>
</ns1:employee>
<ns1:employee>
<ns1:firstname>Thelesa</ns1:firstname>
<ns1:lastname>Lisbon</ns1:lastname>
<ns2:addresses>
<ns2:address location="home" >
<ns2:city postalcode="60000">Madrid</ns2:city>
<ns2:road>Palace street</ns2:road>
</ns2:address>
<ns2:address location="office" >
<ns2:city postalcode="10003">NY</ns2:city>
<ns2:road>cross street</ns2:road>
</ns2:address>
</ns2:addresses>
</ns1:employee>
</ns1:employees>
</soapenv:Body>
</soapenv:Envelope>

And use following JSON message as the sample output and load output message format to data mapper editor.

{
"engineers":{
"engineerList":[
{
"fullname":"Mike Jhonson",
"addresses":{
"address":[
{
"location":"home",
"city":{
"postalcode":30000,
"text":"AA"
},
"road":"BB"
},
{
"location":"office",
"city":{
"postalcode":10003,
"text":"BB"
},
"road":"CC"
}
]
}
},
{
"fullname":"Mike Jhonson",
"addresses":{
"address":[
{
"location":"home",
"city":{
"postalcode":30000,
"text":"AA"
},
"road":"BB"
},
{
"location":"office",
"city":{
"postalcode":10003,
"text":"BB"
},
"road":"CC"
}
]
}
}
]
}
}

Now the editor will have the input and output message formats as bellow.

operations.png

Now we can specify how the output message should build by using the input message. We have several operations given in the operations pallet in the left side.

Click on the operator and it will come to the editor. Then construct the mapping as the following diagram.

mappingdiagram.png

You can only connect primitive data values such as Strings, numbers, boolean and etc. Array and object values can not be mapped.

  • {}  – represent object elements
  • []   – represent array elements
  • <> – primitive field values
  • A   – xml attribute values

After loading the message formats you have to observe whether the formats are correctly identified by the data mapper diagram editor.

Now save the all configurations. We can now deploy the created REST API in the ESB server and test our mapping.

Deploy the REST API in ESB Server

We can create a CAPP including the WSO2 synapse configurations and the registry resource files as below.

DMCAPP.png

Start ESB server and add it to Developer Studio ESB Tool servers and deploy.

deployment.png

Then go to API’s list from the ESB management console and confirm the REST API invocation URL.

deployedAPIs.png

Invoke Created REST API

Open Postman and create the client message as follows.

postmanxmlmessage.png

Send the message from Postman and you will receive the respond message but in the xml format. This is since though we converted the message to type json in the axis2 level we need to change the property “messageType” to “application/json” too.

messagetypeproperty.png

So add a property mediator and configure the following parameters.

  • Property Name : messageType
  • Value : application/json
  • Property Scope : axis2

Then redeploy the CAPP and send the employee message again from the Postman. Then you will receive the expected json message as follows.

jsonresponse.png

{
"engineerList": [
{
"addresses": {
"address": [
{
"city": {
"postalcode": 30000,
"text": "KS"
},
"location": "home",
"road": "main rd"
},
{
"city": {
"postalcode": 10003,
"text": "NY"
},
"location": "office",
"road": "cross street"
}
]
},
"fullname": "Mike Jhonson"
},
{
"addresses": {
"address": [
{
"city": {
"postalcode": 60000,
"text": "MELBORNE"
},
"location": "home",
"road": "park street"
},
{
"city": {
"postalcode": 10003,
"text": "NY"
},
"location": "office",
"road": "cross street"
}
]
},
"fullname": "Patric Jane"
},
{
"addresses": {
"address": [
{
"city": {
"postalcode": 60000,
"text": "MADRID"
},
"location": "home",
"road": "palace street"
},
{
"city": {
"postalcode": 10003,
"text": "NY"
},
"location": "office",
"road": "cross street"
}
]
},
"fullname": "Thelesa Lisbon"
}
]
}

Now you have the basic end to end knowledge to develop a ESB configuration with a data mapper mediator. So you can try this and report the bugs and suggest improvements from the following link as JIRAs.

I will discuss more advanced discussion about how things work inside and mapping configuration is generated from the diagram which will help to understand the behavior of the data mapper and the limitations.

How to Debug WSO2 ESB Mediation Flow

How to Debug WSO2 ESB Mediation Flow

INTRODUCTION

WSO2 Enterprise Service Bus is a lightweight, high performance, and comprehensive ESB. 100% open source, the WSO2 ESB effectively addresses integration standards and supports all integration patterns, enabling interoperability among various heterogeneous systems and business applications.

And now it also contains message mediation debugging support with the release version 5.0.0. In this post I will deploy a simple proxy on WSO2 ESB and debug the message mediation flow.

PREREQUISITES

First we need to have the mediation debugger supported ESB distribution. WSO2 ESB 5.0.0 is the distribution packed with mediation debugger. You can download latest pack from here.

And also we need to have a debugger supported Developer Studio ESB Tool. You can follow this article to install the WSO2 Developer Studio ESB Tool 4.1.0.

OVERVIEW

Lets create a simple scenario where we have a  client, ESB Server and Service.

ESB Debugger Simple USecase(1)

We can use WSO2 try it tool or SOAP UI to send a request from client side. And as this is just to check the ESB Mediation Debugger functionality we will use WSO2 server inbuilt echo service as our service.

Creating Proxy Service

So first now we have to built a proxy service to WSO2 ESB which will get the client request and do some modifications on the received message and sent it to our echo service and get the response from the echo service and sent it back to client after some modifications.

So start WSO2 Developer Studio ESB Tool and create a proxy service with name “EchoServiceWithName”.

It will simply get the following request from the client.

<login>
<user>
<displayname>Mike</displayname>
</user>
</login>

Modify it to following message before sending it to echo service.

<echoString xmlns:ns1="http://echo.services.core.carbon.wso2.org"><in>Mike</in>
</echoString>

And we will receive a message like below.

<echoStringResponse xmlns:ns="http://echo.services.core.carbon.wso2.org">Mike</ns:echoStringResponse>

 

 

Then we will modify it to following message and send it back to client.

<user>
<message>
<welcome>Welcome</welcome>
<name>Mike</name>
<end>for WSO2 technologies !!!</end>
</message>
</user>

This is done by the following configuration.

<proxy name="EchoServiceWithName" startOnLoad="true" trace="disable" transports="http https" xmlns="http://ws.apache.org/ns/synapse">
<target>
<inSequence>
<log/>
<property expression="/login/user/displayname" name="Name" scope="default" type="STRING"/>
<payloadFactory media-type="xml">
<format>
<ns1:echoString xmlns:ns1="http://echo.services.core.carbon.wso2.org">
<in>$1</in>
</ns1:echoString>
</format>
<args>
<arg evaluator="xml" expression="get-property('Name')"/>
</args>
</payloadFactory>
<send>
<endpoint>
<address trace="disable" uri="http://localhost:8280/services/echo"/>
</endpoint>
</send>
</inSequence>
<outSequence>
<log/>
<property expression="/ns:echoStringResponse" name="EchoName" scope="default" type="STRING" xmlns:ns="http://echo.services.core.carbon.wso2.org"/>
<payloadFactory media-type="xml">
<format>
<user>
<message>
<welcome>Welcome</welcome>
<name>$1</name>
<end>for WSO2 technologies !!!</end>
</message>
</user>
</format>
<args>
<arg evaluator="xml" expression="get-property('EchoName')"/>
</args>
</payloadFactory>
<send/>
</outSequence>
<faultSequence/>
</target>
</proxy>

After creating proxy service with above configuration it should be as follows.

proxyDiagramEditor

Now we are all set to debug. We have all components ready to test out proxy service with debugger.

Starting WSO2 ESB Server with Developer Studio ESB Tool to Debug

There are two ways to start a wso2 esb server.

  • First way is from the terminal we can navigate to server file location bin folder and execute wso2server.sh as follows.startserverterminal
  • Second way is we can start esb server from Developer Studio ESb Tool by adding it from the Servers view.serverTab

 

But to support mediation debugging we need to provide command line arguments as “-Desb.debug=true”.

Then the ESB server will listen on two ports to connect with Developer Studio ESB Mediation Debugger as configured in the [esbserver]/repository/conf/sypanse.properties file.

#configuration for the external debugger channels if server is started in debug mode
synapse.debugger.port.command=9005
synapse.debugger.port.event=9006

Then we have to connect developer studio esb tool with esb server by execution above created debug configuration. We have roughly 1 minute time span to connect, otherwise esb server will stop listening and start the server without connecting with debugger tool.

So we should first build the debug configuration from the WSO2 Developer Studio ESB Tool before starting the ESB Server in the debug mode.

Go to Debug Configurations in the Developer Studio and you will see a type of debug configuration named ESB Mediation Debugger

debugconfigclosemediationdebugger.png

Then double click on ESB Mediation Debugger and you will get configuration dialog as below.

defaultconfig.png

It will contain the default command port and event port as 905 and 9006 with your local host name.

If you want to change the default ports, you should modify both server synapse.properties file and this configuration parameters.

And you can remote debug a server which is running in a different host by modifying the Server host with it’s ip.

Now start the ESB Server in the mediation debug mode with the command “sh wso2server.sh -Desb.debug=true” and click the Debug button when server listing to the ports as bellow.

listining.png

Deploy Created Artifacts in Server

Now we have to deploy the previously created proxy service in ESB Server. For that we can create a CAPP using developer studio.

Capp.png

To deploy it from developer studio we need to have the ESB server in the Servers of developer studio. If you started the ESB server from developer studio you can noe directly add the CAPP. Otherwise add the ESB server by going to Servers view-> Add New Server -> WSO2 -> carbon remote server , and specify the host name as follows.newServer.pngremoteip.png

Now you can add and deploy the capp in the server.

addremoveclose.png
deploy.png
serverdeployment.png

Our use case should work. So go to the ESB Server Management Console and select our proxy service try this service tool.

And send the request message as we discussed above. But we will not get the expected output.

Request.pngresponse

Our response doesn’t contain the name. So lets debug the message flow. Put debug breakpoints by right click on the mediators.

breakpoint.png

And send the request again. ESB Message mediation will stop at the first breakpoint position.

debughitpoint.png

There you can see in the variables table under Synapse Scope Properties “Name” property we defined  is empty. There should be an error of out expression. Oh!! yap, it is wrong the expression should be start with double slashes like this “//login/user/displayname” .

Lets look that do we have more errors. If above expression was correct now we have a the value of property “Name” as Mike. So put the value in the variable table and enter. Now the property  value will be changed to “Mike” in the ESb Server.

mikeedit.png

Lets resume the process. It will stop at send mediator. And we will have the expected message after the payload factory mediator. So we can continue.

Oh then another mistake. The returned message from the Echo service is not what we expected.

returnMessage.png

It contain another element named “return” inside the “echoStringResponse” element. So we need to modify the expression of the  property mediator in the out sequence too. It should be “//ns:echoStringResponse/return”.

Lets again put the property value as “Mike” for the property “EchoName” so that we could check the next steps.

modifyedMessage.png

Now we can see the final message is what we expect. So save the modifications done and redeploy the proxy to the server.

And send the request again. We now get the expected response. Great!!!

You will observe that after we redeploy the proxy ESB didn’t suspended at the breakpoints. That is because when the configuration changed in the ESB Server all breakpoint information will be lost. So if we want to debug it we have to re send the breakpoints. For that you only need to right click on ESB diagram editor and in the context menu you will see command Resend ESb Breakpoints. Thats it.

 

resendclose.png

We will discuss more on other features of WSO2 ESB Mediation Debugger in the next post. Since this covers the how to debug ESB Mediation you can also self observe the other features.

You can try this and report the bugs and suggest improvements from the following link as JIRAs.