Adsense

Showing posts with label MuleSoft. Show all posts
Showing posts with label MuleSoft. Show all posts

January 14, 2021

Filter complex array using dw2-Mule4

Hello friends,

In this article I will show you how we can filter results from complex Arrays. In this use-case we will use select and descendent operators from Mulesoft dataweave.

In this use-case we will try to find out the Book details which is at-least rented once from Library. We will see some variations to dataweave expressions in below section.



Input Payload:

[
  {
    "bookId": "20200114",
    "bookType": "Fiction",
    "title": {
      "en": "Candide"
    },
    "message": {
      "en": ""
    },
    "bookDetails": [
      {
        "label": {
          "en": "Candide"
        },
        "Library": {
          "city": "Pune",
          "rented": {
            "count": "1"
          }
        }
      }
    ]
  },
  {
    "bookId": "20200115",
    "bookType": "Fiction",
    "title": {
      "en": "The Alchemist"
    },
    "message": {
      "en": ""
    },
    "bookDetails": [
      {
        "label": {
          "en": "The Alchemist"
        },
        "Library": {
          "city": "Kolkata",
          "rented": {
            "count": "0"
          }
        }
      }
    ]
  }
]

DW Script:
%dw 2.0
output application/json
---
payload[?($.bookDetails.Library.rented.count[0] >= "1")] default []


Output:
[
  {
    "bookId": "20200114",
    "bookType": "Fiction",
    "title": {
      "en": "Candide"
    },
    "message": {
      "en": ""
    },
    "bookDetails": [
      {
        "label": {
          "en": "Candide"
        },
        "Library": {
          "city": "Pune",
          "rented": {
            "count": "1"
          }
        }
      }
    ]
  }
]


Now let us understand the dataweave script in some details:

filter-complex-array-dw-script
DW-Script

We can add multiple conditions inside red brackets based on our requirement.
For example if we want to add filter books based on Library location. We can use
expression like below:

%dw 2.0
output application/json
---
payload[?(($.bookDetails.Library.rented.count[0] >= "1") and ($.bookDetails.Library.city[0] == "Pune"))] default []
%dw 2.0
output application/json
---
payload[?(($.bookDetails.Library.rented.count[0] >= "1") and ($.bookDetails.Library.city contains "Pune"))] default []

Here, important thing to understand about [0] inside red-brackets. Actually we either need
to use [0], because as we have already said in diagram that it iterates and evaluates each
record and give output result or contains operator to evaluate the condition. Also, when
we are having any count check and compare like "> or <" contains operator will not work.


When we are using selectors for filtering data with one more variation in expression.
Let's see some more variation where we can use descendent operator to evaluate
the results.

%dw 2.0
output application/json
---
payload[?(($..count[0] >= "1") and ($..city[0] == "Pune"))] default []
%dw 2.0
output application/json
---
payload[?(($..count[0] >= "1") and ($..city contains "Pune"))] default []


Results will remain same with expression used with descendent and contains operator.

Happy Learning:)
Read More »

January 10, 2021

How to lookup CSV table using Dataweave2

 Hello Friends, 

In this article we will see how to lookup CSV file data without converting it to xml/json or any other message format.

Let's take one example where we need to lookup data from CSV file and retrieve/filter results from the dataset. Here, we will take one very common dataset with employee details information and then we will filter the data with some set of desired condition.


CSV table Input

Name Location Pincode
Niral Pune 910002
Nikku Pune 910005
Shruthi Bhopal 843001
Manpreet Chandigarh 890021
Little John Mumbai 200011
Harry Delhi 100011
Tom Goa 500110

DW Script:

%dw 2.0
output application/json
---
(payload filter ((item, index) -> item.Location == "Pune"))


Output:
[
  {
    "Name": "Niral",
    "Location": "Pune",
    "Pincode": "910002"
  },
  {
    "Name": "Nikku",
    "Location": "Pune",
    "Pincode": "910005"
  }
]


In above example we are filtering data based on location. Here, we have not converted csv data in json before using filter. You can also store the results in dataweave vars and use those results based on your use-cases. 

Please find below some variations having same output as results:
%dw 2.0
output application/json
---
payload filter ($.Location == "Pune")


You can have multiple conditions also. Below expression, we are retrieving data from csv file inside project. You can use readUrl to load csv data from classpath and then use filter  based on Location and Pincode.
 
%dw 2.0
output application/json
var fiterResult= readUrl("classpath://csvFiles/ProfileDetails.csv","application/csv") filter ($.Location == "Pune" and $.Pincode == "910005")
---
fiterResult

Output:
[
  {
    "Name": "Nikku",
    "Location": "Pune",
    "Pincode": "910005"
  }
]



Happy Learning :)

Read More »

December 27, 2020

Pattern Matching in Dataweave Using Match Case

 Bonjour!

Just like any other programming language, In Mule4, dataweave provides match function to achieve the functionality of if-else statements. 

A match expression contains a list of case statements that can optionally have an else statement. Each case statement consists of a conditional selector expression that must evaluate to either true or false.

Let's consider a scenario where you need to update certain fields of incoming payload keeping the rest of the fields as it is, you can use a match and case

for example, the following is the input JSON object:

Input:

{
    "address": {
     "houseNumber" : null,
"street" : "IOC Road",
"pincode" : "111222",
"landmark" : "Statue of unity",
"city" : "Ahmedabad",
"state" : "Gujarat",
"country" : "India"
    }
}

Dataweave Transformation:

%dw 2.0
output application/json
var flag = if (payload.address.houseNumber == null) false else true
---
{
"updateAddressRequest":
payload.address mapObject (value,key,index) ->(
key as String match {
case "pincode" -> {(key): "000000"}
case "houseNumber" -> {(key): if(flag == false) "00" else ""}
else -> {(key):(value)}
}
)
}

Output:


{
"updateAddressRequest": {
"houseNumber": "00",
"street": "IOC Road",
"pincode": "000000",
"landmark": "Statue of unity",
"city": "Ahmedabad",
"state": "Gujarat",
"country": "India"
}
}

What we did here with the input JSON object is that we matched the keys using keyword match and then based on a condition updated the value of the key. We then used the else keyword in order to keep the rest of the fields the same as the input. 

In above example we are updating houseNumer and pincode field with case and other fields are remains same with else block.

This is also an example of how we can use the match case with mapObject

Pattern matching using Match Case can be done on Literal values, on expressions, on Data Types, and on Regular Expressions. 

Adios amigos!



Read More »

December 19, 2020

Mulesoft Interview Questions With Answers - Part 2

Bonjour!

Let's take a look at some more Mulesoft basic interview-related questions!

Which component is used for schema validation and routing incoming Mule messages against RAML?

APIkit Router. 

Mule provides APIkit Router for Schema validation and routing the incoming Mule messages, serializing responses, validating payloads, Query parameters, URI parameters against RAML


Where are the HTTP, database configurations defined?

Global.xml

What is the purpose of a domain project?

The main purpose of a domain project is to have all the globally shared configurations for the projects. All the shared resources can be configured in a domain project. Mule applications can be associated only with one domain project. However, a domain project can be associated with multiple mule applications.


Does Cloudhub support Mule domain projects?

No, Mule domain projects neither supported in Cloudhub nor in Runtime Fabric. Domain projects are supported in on-premises environments.


Which connector is used for handling SOAP requests?

Web service Consumer.

How we can invoke REST or SOAP Services?

To invoke REST we can use HTTP and the Web service Consumer connector can be used to invoke SOAP services.


What is the use of Message Enricher?

In Mule 3, Message enricher is a component used to enrich the current message with additional information from a separate resource without disturbing the current payload.

Enricher is used in scenarios where the target system needs more information than the source system can provide. It calls an external system or doing some transformation to the current payload and saving it in a target variable.


Do we have Message Enricher in Mule 4? 

In Mule 4, We don't have a message enricher anymore. 
Now, every component is embedded with a target variable which solves the purpose. 
 

How do you override application properties?

You can configure your Mule application to automatically load properties from properties files based on different environments (Production, development, testing, QA) using property placeholders. 

1.  Create a properties file for each development environment in your Mule application.
2. Configure a property placeholder in your application to look for the environment at the time of launch.
3. Set environment variable one each for the different environment.

The property placeholder will then take up an environment variable value and based on that use the properties file.


How can we store sensitive data such as Client ID and Client Secret? 

Properties in Mule 4 can be encrypted to store sensitive data such as Client ID and Client Secret. We can encrypt a .yaml or .properties file using the secure properties module. For more details about the use of  secure properties: securing-configuration-properties-in-mule


Give some examples of popular Mulesoft connectors? 

HTTP, File, FTP, SFTP, Database, Salesforce, SAP, AWS, etc.


Can you create a custom connector? 

Yes. 

When an API specification is published to Anypoint Exchangeit automatically generates a connector for it. You can find out a list of Mulesoft connectors here.


Advantages of Mulesoft Anypoint connectors? 

  • It provides integration of Mulesoft application with different 3rd party systems like Facebook, Twitter, Salesforce, etc.
  • It is easier to maintain and no need to spend time on optimization and security.
  • It provides reusability which helps in speed-up development.


Is Mulesoft PaaS or SaaS? 

Mulesoft provides iPaaS (Integration Platform as a Service). It provides connectivity across on-premises and SaaS applications.


In Mule4 how you can generate custom errors.

By using the Raise Error component


In Mule 4, how errors are handled in a mule flow?

When an error is thrown in Mule flow while processing an event first step is it stops normal flow execution and it will not go to the next components. Then depending on the error type either error will be caught and handled inside the error handling block or thrown up to its parent flow.


Where we can add an error handler in the Mule 4 application?

An error handler can only be added to

– A flow

- A private flow

– A Try scope

– An global error handler (outside of any flows) is then referenced to any of the error handling strategies.

Note: Error handler can not be added to subflows.


What is the Mule default error handler?

When there is no error handler defined for any flow or by using try scope, the Mule default error handler will be used. This default handler should be configured globally.


What if the default error handler not configured in the Mule application?

If the default error handler is not configured then Mule runtime will provide its own default behavior

– First, it stops the execution of the flow and logs information about the error.

– Implicitly and globally handles all errors thrown in Mule applications.

– It automatically rethrows the error.

– Most important we cannot configure it.


What are error scopes in mule4 to handle errors?

– On Error Continue scope

– On Error Propagate scope

 

Can you please explain the difference between these two error scopes in Mule4?


What is the use of Try scope in Mule4?

Try Scope is very useful when you need to group a set of events/ components and want to handle errors based on your specific requirements. It acts as a private flow inside the parent flow. For Unhandled errors, it propagates to its parent flow.



Read More »

December 12, 2020

Mulesoft Interview Questions With Answers - Part 1

Bonjour!

What is Mulesoft?

The first and the most basic question asked in Mulesoft developer interviews. 

Mulesoft is the vendor that provides an integration platform to help businesses connect data, applications, and devices.

Mule is the runtime engine of the Anypoint Platform. It is a lightweight Java-based enterprise service bus (ESB) that allows developers to connect applications and exchange data. It can connect any systems by using different protocols like HTTP, JMS, JDBC etc.


What is Anypoint Platform?

Anypoint platform is a hybrid integration platform which is highly productive, unified and helps to create a smooth distributed systems. It helps to manage full API lifecycles. It provides unified platform to deploy, manage, monitor and rapid development of apps into single place.


What is a Mule Application?

A Mulesoft application is a package of codes and files that runs inside Mule Runtime. Mulesoft application can be triggered with set of events that can be external and inside mulesoft. It can process events and route to next components or endpoints. Endpoints can be external system, another mulesoft application or local resources like file.

A Mule Application may consists of one or more flows, subflows and private flows.


What is the difference between Flow, Subflow, and Private flow?

Flow is a message/event processing block. It can have Mule sources (e.g. an HTTP listener) to trigger the execution of a flow. It can have its own processing strategy and error handling strategy.

Subflow is a scope that enables you to process messages synchronously. It doesn't have any source or error handling block. It inherits the processing strategy and error handling strategy from parent flow. It can be used to split common logic and reusability.

Private flow is a flow without a Mule source. It can be synchronous or asynchronous based on the processing strategy selected. They can have their own error handling strategy.

All three above can be called using a flow-ref. Source will be skipped while calling using flow-ref.


How Mule Applications can be build?

Mule application can be created using online flow-designer and offline by downloading Anypointsudio. 

Link for Online flow-designer

Link to download AnypointStudio


How Mule4 Application flow executes?

Mule4 application flows executes events in multiple threads. Mule4 automatically optimizes performance and uses thread switching and parallel processing if required.


Variables in Mule 4 vs Mule 3?

In Mule 3, we had three types of variables: Session variables, Record variables, and Flow variables.

In Mule 4, we only have Flow variables. These variables can be accessed using the keyword "vars".

Example:

vars.variableName


What is API-led connectivity?

API-led connectivity is a methodical way to connect applications to data through reusable APIs. APIs are created in three distinct layers: Experience, Process, and System.




API Life Cycle?

An API life cycle consists of 3 layers:



The first step towards API development is API Specification. This phase involves designing of the API based on the requirements of business. An API Specification is a blueprint of an API.

Anypoint Platform provides Design center to design API's REST interfaces.

The next phase is API Implementation. This phase involves actual implementation of the API.

Anypoint Platform provides Flow Designer which is a web based tool to create a prototype of the API.

Anypoint Studio is the IDE for development and testing of Mule applications. 

Final phase in API life cycle is API management, uses a tool named API manager on Anypoint Platform which is used to apply policies to secure APIs and restrict access. 


What are the advantages of API-led approach?

  • Mulesoft prefers Design first approach for API development that helps non-technical stakeholders easy to understand the API specs/contracts.
  • It simplifies the interface between Client and application by hiding complex details.
  • Business validation and contracts can be defined at API specification. So, developers don't need to invest time to write code for validation and error type checks.
  • It encourages "fail fast" approach which helps in fast and parallel development.
  • It encourages reusability of assets and collaboration across teams.
  • Apis can be mocked before implementation starts which helps early feedback and parallel development.
  • Mulesoft anypoint platform provides tools to write, publish, share and manage versions of API.


What is RAML?

RAML stands for RESTful (REpresentational State Transfer) API Modeling Language. It's a way of describing a RESTful API that is readable by both human and computers. 

They use https methods such as GET, PUT, POST, DELETE, etc.


Difference between URI parameters and Query parameters.

URI parameter is used to specify a specific resource whereas Query parameter is used to filter or sort those resources.

let's say we need to filter products based on productType, we use query parameters in this case:

GET: /products?productType=electronics

Let's consider another scenario where we need to retrieve particular product details, we use URI parameters in this case:

GET: /products/{productID}

here, productID will be replaced by a value.


Error Handling in Mule 4?

Error handling in Mule 4 is redesigned. It has three mechanisms to handle errors now.

1. On-Error Propagate:  In an On-error Propagate scope, when an error occurs, it terminates the current execution of flow and then throws the error to its parent flow where it's handled.

2. On-Error Continue: In case of an error in an On-error Continue scope, the execution of the flow continues as if successful sending a 200 OK response.

3. Try Catch scope: A Try Catch scope is used to handle errors on individual components. It is especially useful when you need to separate error handling strategies for different components in a flow.


What are the Logger levels?

There are 5 logger levels namely: DEBUG, ERROR, INFO, TRACE, and WARN. 


What does scatter-gather return?

A scatter-gather returns a Mule event.


What is the use of Async scope?

Async scope can be used to trigger flows subflows or any components asynchronously from a parent flow. Async scope is very helpful while using logger component or parallel processing requires.


What is the difference between map and mapObject?

map operator is used for array. It takes an array as input and returns an array.

mapObject operator is used for object with key: value pairs. It takes an object as input and returns an object.


map invokes lambda with two parameters: value and index from the input array. 

Example:

%dw 2.0
output application/json
---
["apple", "orange", "banana"] map (value, index) -> { 
    (index) : value}

Output:

[ { "0": "apple" }, { "1": "orange" }, { "2": "banana" } ]


mapObject invokes lambda with keyvalue, or index for mapping given object.

Example:

%dw 2.0
output application/json
---
{"chandler":"bing","monica":"geller"} mapObject (value,key,index) -> { 
    (index) : { (value):key} }

Output:

{ "0": { "bing": "chandler" }, "1": { "geller": "monica" } }



What is batch processing?

Mule allows you to handle large quantities of data in batches. 

This is achieved using a Batch job scope which splits messages into individual records, performs actions on records, and reports the results. A Batch job scope processes data asynchronously.


Each batch job contains three different Processing Phases:  

1. Load and Dispatch: In this phase, Mule splits the message, creates a Batch Job Instance, a persistent queue and associates it with the Batch Job Instance.

2. Process: This phase does the actual processing of records asynchronously. 

3. On Complete: This is an optional phase where you can configure the runtime to create a report or summary of records. 


Stay tuned for more!





Read More »

February 08, 2020

Optimize Json data using DataWeave

In this Article, we will see how we can optimize and allow Json data to accept and transform duplicate elements. In real-time projects optimizing data really matters when we talk about tuning performance. Either you say in terms of  logging data or in terms of processing data every byte matters.


In DataWeave, we have two properties which will hep us to optimize our application data.

  • duplicateKeyAsArray: JSON data does not allow duplicate keys within same parent, so we may get exception in real-time project development scenario. With this attribute, if your dataweave finds any duplicate keys, it will create an array with all those values.

Sample XML Input:

<users>
 <user>
  <personal_information>
   <first_name version="V1">Manish</first_name>
   <first_name version="V1">Mak</first_name>
   <middle_name>Kumar</middle_name>
   <last_name>Sah</last_name>
  </personal_information>
 </user>
</users>


Dataweave Script:


%dw 2.0
output application/json duplicateKeyAsArray=true
---
payload


Sample Output:


{
  "users": {
    "user": {
      "personal_information": {
        "first_name": [
        "Manish",
        "MaK"
        ],
        "middle_name": "Kumar",
        "last_name": "Sah"
      }
    }
  }
}

  • indent: It indicates whether you want to compress your JSON data into a single line or you need JSON data for better readability.
Sample Input:
<users>
 <user>
  <personal_information>
   <first_name version="V1">Manish</first_name>
   <first_name version="V1">Mak</first_name>
   <middle_name>Kumar</middle_name>
   <last_name>Sah</last_name>
  </personal_information>
 </user>
</users>


Dataweave Script:


%dw 2.0
output application/json duplicateKeyAsArray=true, indent=false
---
payload


Sample Output:


{"users": {"user": {"personal_information": {"first_name": ["Manish","MaK"],"middle_name": "Kumar","last_name": "Sah"}}}}


Happy Learning :)

Read More »

February 06, 2020

How to write Attributes from XML to Json - DW2

In project development sometimes we needed to convert the payload from xml to json. We also need to write the xml attributes after transformation. 

For this purpose we can use writeAttributes=true in header section of data-weave. This attribute is available from Dataweave2.


Please find the snippets below:



Sample XML Input:

<users>
<user>
<personal_information>
<first_name version="V1">Manish</first_name>
<middle_name>Kumar</middle_name>
<last_name>Sah</last_name>
</personal_information>
</user>
</users>


Dataweave Script

%dw 2.0
output application/json writeAttributes=true
---
payload

Output:

{
  "users": {
    "user": {
      "personal_information": {
        "first_name": {
          "@version": "V1",
          "__text": "Manish"
        },
        "middle_name": "Kumar",
        "last_name": "Sah"
      }
    }
  }
}

Happy Learning :)
Read More »

December 05, 2019

Anypoint MQ Circuit Breaker - Mule4

In integration development there are lot of patterns that we can follow based on project requirements. Circuit Breaker is one of them.
In this article we will see how Circuit Breaker can be used and what we should consider while implementing it. 

Circuit breaker design pattern is used when:

  • To handle error scenarios and message reprocessing strategy.


Mulesoft has provided circuit breaker capability with Anypoint MQ component.
You can add dependency inside your pom as below:

<dependency>
<groupId>com.mulesoft.connectors</groupId>
<artifactId>anypoint-mq-connector</artifactId>
<version>3.1.0</version>
<classifier>mule-plugin</classifier>
</dependency>

Now, I will show you how you can design and configure your mule flows for implementing Circuit Breaker capability.


Below is the sample circuit breaker mule flow design:


flow-design-circuit-breaker

We will explain each and every part the above flow design below:


Circuit Breaker Configuration:

Go to global elements section create a Circuit breaker configuration as below:



circuit-breaker-configuration


This configuration can be shared across multiple Subscriber resources in the project.
On Error Types – Type of Errors based on which you want to Open the Circuit.
Errors Threshold – No. of Failures in Succession – Ex: 5
Trip Timeout – Duration to keep the Circuit Open once the ErrorThreshold is reached – Ex: 60 Minutes


Subscriber Configuration

Anypoint MQ Subscriber Source provides Circuit Breaking Capability.
                                                       
subscriber-circuit-breaker-config

 
1. Select the Anypoint MQ Subscriber Source.
2. Click on the Advanced tab.
3. Provide the Circuit Breaker from Global Reference.



AMQ Message Processing\Listening Strategy:


amq-subscriber-configuration


This section describes the methodology how we are going to follow to process messages from Anypoint MQ. This process helps in effectively handling reprocessing and retrying scenarios and also simplifies the process from a maintenance perspective.


Steps:

  1. Configure the Subscribe processor
  2. Subscriber Type – Prefetch
  3. Max Local Messages –  default it to 5, you can change based on requirement for further tuning.
  4. Acknowledgment Mode – Set to MANUAL, As the Acknowledgement Mode is set to MANUAL, we need to Ack the message post successful completion of the flow. So that the message doesn’t get retried/consumed.


Circuit Breaker and AMQ Listening Strategy both need to be configured in our components where we want messages to be reprocessed and retried in a configurable manner.

In case of Error Scenarios,



error-propagate-type

Based on your requirement you can catch the error and subsequent to Error Handling, Nack the message so that the message stays on the queue and will be subsequently redelivered.


nack-token-vars



You can use Ack the message so that the message is removed from the queue in case of success scenarios.


ack-token-vars

This ackToken you can store in variable which can be used for Ack/Nack of the message.


set-attributes


MQ messages stores in queue after getting Nack

mq-msg-properties-details



Please find sample Mule project in Github Error-Handling-using-Anypoint-MQ-Circuit-Breaker
Read More »