Category: Uncategorized

How To Debug WSO2 ESB in Different Tenants

How To Debug WSO2 ESB in Different Tenants

INTRODUCTION

WSO2 Enterprise Service Bus is a lightweight, high performance, and comprehensive ESB. 100% open source, the WSO2 ESB effectively addresses integration standards and supports all integration patterns, enabling interoperability among various heterogeneous systems and business applications.

And now it also contains message mediation debugging support with the release version 5.0.0. In this post I will deploy a simple proxy on WSO2 ESB and debug the message mediation flow.

PREREQUISITES

First we need to have the mediation debugger supported ESB distribution. WSO2 ESB 5.0.0 is the distribution packed with mediation debugger. You can download RC2 pack from here.

And also we need to have a debugger supported Developer Studio ESB Tool. You can follow this article to install the WSO2 Developer Studio ESB Tool 5.0.0

If you have never tried the wso2 mediation debugger before follow previous post to understand the basics to use debugger.

============================================================================

WHAT ARE TENANTS

The goal of multitenancy is to maximize resource sharing by allowing multiple organizations (tenants) to log in and use a single sever/cluster at the same time, in a tenant-isolated manner. That is, each user is given the experience of using his/her own server, rather than a shared environment. Multitenancy ensures optimal performance of the system’s resources such as memory and hardware and also secures each tenant’s personal data.

You can register tenant domains using the Management Console of WSO2 products.

Please follow following two articles to know more about WSO2 tenant architecture and how to use it.

  1. WSO2 Multi tenant Architecture
  2. Managing tenants in WSO2 products

How To Debug Tenants

If you have followed my previous post you know that first we need to start the esb in debug mode. For that we need  to use command -Desb.debug=true. Now we have changed this and you need to use -Desb.debug=$tenantDomain.

So for example if you need to debug the super tenant (It is the default one. If you have no idea about a tenant, that means you are working as super tenant 🙂 ) you can use the command -Desb.debug=super or -Desb.debug=true.

  • Start Debugger in super tenant : -Desb.debug=super

Since the esb server will start in the super tenant mode, the server starting will be suspend till you connect the ESB Tool with server on two ports. So what you need to understand is to connect the server with tool to enable debugging you need to start the server in that specific tenant mode. If you are trying to debug a tenant which is not the super tenant, you may not get the server listening state as it for super tenant. That is because the lazy loading behavior of the wso2 tenants.

[Note-Lazy loading is a design pattern used specifically in cloud deployments to prolong the initialization of an object or artifact until it is requested by a tenant or an internal process.]

Lazy loading of tenants is a feature that is built into all WSO2 products, which ensures that in an environment with multiple tenants, all tenants are not loaded at the time the server starts. Instead, they are loaded only when a request is made to a particular tenant.

So server will start listening to connect with tool when the first request is made for that specific tenant. Lets say you have defined a tenant domain as “foo.com”. You need to start the server as -Desb.debug=foo.com. Server will start normally in the super tenant mode. Then send a request to proxy/inbound/API in the foo.com domain as “http://%5Byour-ip:port%5D/t/foo.com/%5Byour-service%5D”. Then the server will start to listen in the two ports to connect with ESB tool to debug.

  • Start in foo.com tenant domain :-Desb.debug=foo.com

Let’s do a simple scenario to do this. First we need to create a tenant domain. Start esb server and go to configure section in the management console.

esbconfiguretab.png

You will see Multi-tenancy section at the bottom. Click on Add new Tenant.

addnewdomain.png

Configure the required fields. And sign in to esb server using entered username and password.

  • User name  : adminfoo@foo.com
  • Password    : ******       🙂

Then deploy your artifacts to this tenant. And you will get endpoint urls for your tenant domain foo.com.

I have created very basic artifact API to test this scenario.

sampleArtifact.png

To deploy it to foo.com tenant add the started server as a remote server and use tenant credentials.

tenantserver.png

Then add the capp to server and deploy it.

artifacts_deployed.png

Now shutdown the server and start it with command sh wso2server -Desb.debug=foo.com.

serverStarts.png

So the server will start in the super tenant mode. Now send a request to our deployed API in the tenant.

sampleRequest.png

Now you will observe that the server starting to listen on two ports to connect with ESB tool.

serverListens.png

Now connect with server form the ESB tool.

connectWithserverTodebug.png

Then resend/send breakpoints to server.

resendESBBreakpoints.png

And send the request again. ESB will suspend on the breakpoint.

debuggerInvoked.png

What if you do not want to wait to connect the tool when the first request comes. You need to do it in the server start up. Can you do it???

Of course you can. 🙂

What you need to do is disable the Lazy loading of the server and enable the Eager Loading.

How can you do it?

Go to [ESB_HOME]/repository/conf/ and open the carbon xml file. Go to Tenent/LoadingPolicy configuration and comment the LazyLoading policy and uncomment the EagerLoading policy to start all tenants or just foo.com.

 

enableEagerLoading.png

Now the server will suspend and listen on ports to connect with ESB tool debugger at the server startup.

So I now you can debug different tenants too… Happy debugging!!! 🙂

 

 

WSO2 Data Mapper Common Type Operators

WSO2 Data Mapper Common Type Operators

INTRODUCTION

WSO2 data mapper is a data mapping solution that convert and transform one format of data to a different format. It provides a WSO2 Developer Studio based tool to create the graphical mapping configuration and generate the files needed to execute the mapping configuration by the WSO2 Data mapper engine.

If you first came to here look for the two previous posts which will give you the basic understanding of WSO2 Data Mapper and how to integrate it with WSO2 ESB.

  1. How to Use WSO2 Data Mapping Mediator in ESB (Updated for-ESB-5.0.0-Beta2)
  2. Understanding WSO2 Data Mapper 5.0.0

OVERVIEW

This post is one of many to introduce data mapper operations available and how to use it in mapping configuration. There are mainly six categories in the operator pallet as shown in the following diagram.

diagramEditor

They are,

  • Links
  • Common
  • Arithmetic
  • Conditional
  • Boolean
  • Type Conversion
  • String

In this post we will discuss about Common Operators, given in data mapper.

Constant Constant- define string, number or boolean constant values.

This operator is basically to define values we need to use in mapping. When you drag and drop the constant operator from the pallet to editor area you will get it as following diagram.

constantdefault

As you can see it only has a output connector. And that value can be a Number, String or Boolean. To configure it right click on constant operator and click the configure constant operator menu action. It will pop up a dialog box to configure the operator as follows.

configuremenuconstant.png

So you can define the value you need in here.

CustomFunction Custom Function – define custom function to use in mapping

Custom function is for more advanced users who need more functionality other than offered in the data mapper. And if you know Java Script well you could do many things by using custom functions with global variable operator. But here I will only describe the basic features and how to use this operator to define our own functionality.

When you drag and drop custom function operator for editor you will get the following default operator and the configuration.

defaultcustomfunctionconfiguratoncustomfunction

You should first give a name to your function in the Function Name text box. You can not use same name to two functions in the same mapping.

Then define the number of parameters/inputs you need in the Number of Inputs text box.

Then you need to define the parameters inside the function definition () according to the number of inputs you specified above. Use those parameters or any Global Variable value inside the custom function to do the functionality and return the value.

Properties Properties – use product specific runtime variables

This operator is discussed in great detail in the following post by Eranda.

GlobalVariable Global Variable – instantiate global variable to access from any where

Global variable operator is given to instantiate a variable and use in any where in the mapping. When you drag and drop the operator to editor you will see following default operator and the configuration.

GVoperatorGVconfiguration

This is basically a JavaScript variable defined in the outer scope so that any one can see and access it. You could use this variable in any custom function you define in the mapping. In the configuration you can change the name and default value of it.

Compare Compare – compare two inputs in the mapping

This operator can be used to compare any two values in the mapping. There are 8 different compare operations you could do. They are,

  • equal value (two inputs are equal or not)
  • equal value type (two inputs are equal in both value and type)
  • not equal value (two inputs are not equal or not)
  • not equal value type (two inputs are not equal in both value and type)
  • greater than
  • greater than or equal
  • lower than
  • lower than or equal

compareOperatorcompareconfiguration

 

 

How to Use WSO2 Data Mapping Mediator in ESB (Updated for-ESB-5.0.0-Beta2)

How to Use WSO2 Data Mapping Mediator in ESB (Updated for-ESB-5.0.0-Beta2)

INTRODUCTION

WSO2 data mapper is a data mapping solution that convert and transform one format of data to a different format. It provides a WSO2 Developer Studio based tool to create the graphical mapping configuration and generate the files needed to execute the mapping configuration by the WSO2 Data mapper engine as shown in the below diagrams. As the this is really useful feature in message mediation, WSO2 ESB comes with a data mapper mediator which can be integrate into a mediation sequence since WSO2 ESB 5.0.0.

PREREQUISITES

First we need to have the data mapper mediator supported ESB distribution. WSO2 ESB 5.0.0 is the distribution packed with data mapper mediator. You can download beta2 pack from here.

And also we need to have a data mapping editor supported Developer Studio ESB Tool. You can follow this article to install the WSO2 Developer Studio ESB Tool 5.0.0.

For this post I will use 5.0.0-beta2 packs of wso2 esb runtime and tooling, which are the latest packs. So the operations mentioned here will be only available with 5.0.0 beta2 or onwards.

We will use the Postman as out client to send the requests to esb and receive responses.

OVERVIEW

Since our main focus is to check the functionality of the Data Mapper we will use a very simple configuration in the esb with a data mapping mediator and a respond mediator. We will check following message conversions,

  • XML->XML/JSON/CSV
  • JSON->XML/JSON/CSV
  • CSV->XML/JSON/CSV

To understand the data mapper further and to examine the operators provided in the data mapper, which we can use to map elements from input to output look in to Understanding WSO2 Data Mapper post.

CREATE ESB CONFIGURATION PROJECT

First this we would need to do is create esb configuration to contain the data mapper mediator. ESB Tooling now provides a new option to create a ESB Solution Project so that you can define the all different projects you need for the project in the wizard. Right click on the Project Explorer area and go to new-> ESB Solution Project

ESBSolutionProject

You will get the following wizard page and give project name and unpick the Connector Exporter Project since we are not going to need connector in our configuration.

ESBSolutionProjectWizard

And you will get the following project files created in your workspace.

ProjectExplorer

Now what we will do is create a new REST Api project in ESB and configure it to listen for POST requests on url “/convertMenu”.

RESTAPIfoodMenuAPI

Then create the API resource with data mapper mediator and listening to POST request.

createAPIAndDataMapperapiPropertiesTable

To configure data mapping mediator double click on the mediator and you will get a dialog box asking to create a registry resource project as below.

createMappingFiles

This name would be the prefix for the configuration files we will deploy in esb server related to data mapper. Since we created a ESB Solution Project we were directly pointed that project in here otherwise go to Create new project link and create a new Registry Resource project.

Then click OK after giving the prefix and pointing the Registry resource project to save it.  Following diagram editor will appeared in new perspective named “WSO2 Data Mapper Graphical”. You can switch this perspective as you wish by selecting in top tool bar tags or Window->perspective->Open Perspective->Other

dataMappingEditor

In the left hand side you can see the operation pallet and in the middle input output boxes. We need to provide the input and output message formats to begin the mapping. So right click on the top title bar of the input box and select command Load Input.

loadInputinputType

As you can see in the above image, there are several options for you to load the input and output message formats to data mapper.

You can select the resource type as XML, JSON, CSV ,XSD ,JSONSCHEMA or CONNECTOR.

If you selects XML, you can load a sample xml message you expect and WSO2 Data Mapper Editor will generate the JSON schema to represent the xml according to the WSO2 Data Mapper Schema specification.

Same as for XML, if you select  JSON/CSV as resource type you can load a sample JSON/CSV message. For CSV you need to provide the column names as the first record.

If you have a xsd schema which defines you xml message format, you can select the resource type as XSD and load the xsd file.

And also if you have the JSON schema for your message according to the WSO2 Data Mapper schema specification you can load it by selecting resource type as JSON schema.

Connector type will come if you are going to map message which is an output of a connector, in the input box you can select connector type and it will list down available connectors and then select the operation in-front of data mapper. This is not the time to explain the connector integration with data mapper. We will discuss it later in a different post. 🙂

For this moment we will load from a sample XML file.

Input xml sample for food menu.

<?xml version="1.0" encoding="UTF-8"?>
<breakfast_menu>
<food>
<name>Belgian Waffles</name>
<price>$5.95</price>
<description>Two of our famous Belgian Waffles with plenty of real maple syrup</description>
<calories>650</calories>
<orgin>Belgian</orgin>
<veg>true</veg>
</food>
<food>
<name>Strawberry Belgian Waffles</name>
<price>$7.95</price>
<description>Light Belgian waffles covered with strawberries and whipped cream</description>
<calories>900</calories>
<orgin>Belgian</orgin>
<veg>true</veg>
</food>
<food>
<name>Berry-Berry Belgian Waffles</name>
<price>$8.95</price>
<description>Light Belgian waffles covered with an assortment of fresh berries and whipped cream</description>
<calories>900</calories>
<orgin>Belgian</orgin>
<veg>true</veg>
</food>
<food>
<name>French Toast</name>
<price>$4.50</price>
<description>Thick slices made from our homemade sourdough bread</description>
<calories>600</calories>
<orgin>French</orgin>
<veg>true</veg>
</food>
<food>
<name>Homestyle Breakfast</name>
<price>$6.95</price>
<description>Two eggs, bacon or sausage, toast, and our ever-popular hash browns</description>
<calories>950</calories>
<orgin>French</orgin>
<veg>false</veg>
</food>
</breakfast_menu>

Output xml sample for food menu.

<?xml version="1.0" encoding="UTF-8"?>
<menu>
<item>
<name>Belgian Waffles</name>
<price>$5.95</price>
<calories>650</calories>
<orgin>Belgian</orgin>
<veg>true</veg>
<description>Two of our famous Belgian Waffles with plenty of real maple syrup</description>
</item>
<item>
<name>Strawberry Belgian Waffles</name>
<price>$7.95</price>
<calories>900</calories>
<orgin>Belgian</orgin>
<veg>true</veg>
<description>Light Belgian waffles covered with strawberries and whipped cream</description>
</item>
<item>
<name>Berry-Berry Belgian Waffles</name>
<price>$8.95</price>
<calories>900</calories>
<orgin>Belgian</orgin>
<veg>true</veg>
<description>Light Belgian waffles covered with an assortment of fresh berries and whipped cream</description>
</item>
<item>
<name>French Toast</name>
<price>$4.50</price>
<calories>600</calories>
<orgin>French</orgin>
<veg>true</veg>
<description>Thick slices made from our homemade sourdough bread</description>
</item>
<item>
<name>Homestyle Breakfast</name>
<price>$6.95</price>
<calories>950</calories>
<orgin>French</orgin>
<veg>false</veg>
<description>Two eggs, bacon or sausage, toast, and our ever-popular hash browns</description>
</item>
</menu>

Load above files as input and outputs and look the input output boxes with sample message. Is the element types are correctly identified or not. (Arrays, Objects and Primitive values). Following signs will help you to identify it correctly.

  • {}  – represent object elements
  • []   – represent array elements
  • <> – primitive field values
  • A   – xml attribute values

Now use operators and do the mapping as you desire. You can only connect primitive data values such as Strings, numbers, boolean and etc. Array and object values can not be mapped.

mapping

What I have done here is name is mapped via uppercase operator and calories undergoes and mathematical calculation to the output.

output calories =Round( (calories*1.13) + 6.75)

Now save the all configurations. We can now deploy the created REST API in the ESB server and test our mapping.

To deploy our configuration we need to include these in a CAPP project. When we created the ESB Solution Project we created the CAPP project too. So open the CAPP project and double click on the pom file. You will get the following page to select project files into CAPP.

CAPPProject.png

Here you do not see the registry resource files. We need to refresh the page to see it. Use refresh button in the top right conner to load newly added registry files.

refreshBUtton

Then you will see the all artifacts in the workspace. Select REST API file and the three registry resource files containing mapping configuration , input schema and output schema.

cappProjectselection

 

  • Mapping Configuration : This file contain the script file used to execute the mapping
  • Input Schema: JSON schema which represent input message format
  • Output Schema: JSON schema which represent the output message format

And also see the data mapper mediator configuration in the REST API.

DMMconfig.png

 

  • Input Type : Expected input message type (xml/json/csv)
  • Output Type: Target Output message type (xml/json/csv)

If your mapping failed at the runtime the first thing you need check whether the mediator configuration. Is the input type and output type are set correctly.

Now start WSO2 ESB server and connect it to ESB tool as a remote server or start ESB server from inside the ESB tool. And add out CAPP to the server and deploy it.

addwso2servercappAdding

Then goto management console from https://localhost:9443/carbon

See the deployed API and get the url it is listening.

APIDeployed.png

Start the POST Man and send the request for the above url.

postMan.png

You will get the response with accordingly to post man when you send the message.

To see the XML -> JSON conversion  just go to REST API  configuration and data mapper properties. Select Output Type as JSON . Deploy the configuration and send the request. You will get the JSON message.

Do the same for see the XML->CSV.

But this is not guaranteed every occasion. If t you have defined complex XML output message with name spaces and attributes JSON message or CSV will not built as you will imagine. In this case output message format is fully compatible to represent as JSON and CSV.

Now you have the basic end to end knowledge to develop a ESB configuration with a data mapper mediator. So you can try this and report the bugs and suggest improvements from the following link as JIRAs.