How to exclude files from deployment - Mule4

Hello Friends,

In this article, I want to show to you how you can skip or exclude flow from initialize and start while deploying mule 4 applications in local or in cloud. Also, I will show you how you can exclude files from packaging jars.

Problem: 

In mulesoft development there are high chance of having large number of files for flows/ subflows and due to this it leads to:

• Long build and deployment time.
• Bulk packaging.

Above scenarios are more painful when we need a quick fix and testing of code in local or in cloud environment.


Let's check scenario below, as we are able to see in project below we have three files under src/main/mule package. Each file is having individual flows as highlighted in picture below:

mule4-flows

In mule4, we have "mule-artifact.json" file where we can declare files to be considered for initializing and starting of the flows under configs tag. By default, if we don't declare any file names, Mule runtime includes All files to check, initialize and starting of the flows.

mule-artifact-json.png

Below logs shows only "HelloWorldFlow" is initialized and started for deployment.

flow-logs

So, if we don't declare any file to include for execution, below logs shows mule runtime started and initialized all flows from all files.

flow-logs

Although above configuration helps you to exclude flows to initiate and start while deploying application in local or in cloud. It helps to reduce total deployment time of mule apps. But with above configuration you will not be able to exclude files or folders from packaging jar.

How to exclude files or folders from packaging jars?

To achieve this, we need to create "_muleExclude" file at the root level of mule application.

_muleExclude-mule4.png

You can declare file names to exclude from packaging jar as shown above.

Above picture also shows extracted jars is not having files included inside "_muleExclude" file.

Note: Please make sure above configurations should be used wisely as it may lead to application failure.

If you want to know more about minMuleVersion you can check my article runtime-patching-in-mule-munit


Happy Learning :)

Read More »

Filter complex array using dw2-Mule4

Hello friends,

In this article I will show you how we can filter results from complex Arrays. In this use-case we will use select and descendent operators from Mulesoft dataweave.

In this use-case we will try to find out the Book details which is at-least rented once from Library. We will see some variations to dataweave expressions in below section.

Input Payload:

[
  {
    "bookId": "20200114",
    "bookType": "Fiction",
    "title": {
      "en": "Candide"
    },
    "message": {
      "en": ""
    },
    "bookDetails": [
      {
        "label": {
          "en": "Candide"
        },
        "Library": {
          "city": "Pune",
          "rented": {
            "count": "1"
          }
        }
      }
    ]
  },
  {
    "bookId": "20200115",
    "bookType": "Fiction",
    "title": {
      "en": "The Alchemist"
    },
    "message": {
      "en": ""
    },
    "bookDetails": [
      {
        "label": {
          "en": "The Alchemist"
        },
        "Library": {
          "city": "Kolkata",
          "rented": {
            "count": "0"
          }
        }
      }
    ]
  }
]

DW Script:

%dw 2.0
output application/json
---
payload[?($.bookDetails.Library.rented.count[0] >= "1")] default []

Output:

[
  {
    "bookId": "20200114",
    "bookType": "Fiction",
    "title": {
      "en": "Candide"
    },
    "message": {
      "en": ""
    },
    "bookDetails": [
      {
        "label": {
          "en": "Candide"
        },
        "Library": {
          "city": "Pune",
          "rented": {
            "count": "1"
          }
        }
      }
    ]
  }
]

Now let us understand the dataweave script in some details:

DW-Script

We can add multiple conditions inside red brackets based on our requirement.

For example if we want to add filter books based on Library location. We can use

expression like below:

%dw 2.0
output application/json
---
payload[?(($.bookDetails.Library.rented.count[0] >= "1") and ($.bookDetails.Library.city[0] == "Pune"))] default []

%dw 2.0
output application/json
---
payload[?(($.bookDetails.Library.rented.count[0] >= "1") and ($.bookDetails.Library.city contains "Pune"))] default []

Here, important thing to understand about [0] inside red-brackets. Actually we either need

to use [0], because as we have already said in diagram that it iterates and evaluates each

record and give output result or contains operator to evaluate the condition. Also, when

we are having any count check and compare like "> or <" contains operator will not work.

When we are using selectors for filtering data with one more variation in expression.

Let's see some more variation where we can use descendent operator to evaluate

the results.

%dw 2.0
output application/json
---
payload[?(($..count[0] >= "1") and ($..city[0] == "Pune"))] default []

%dw 2.0
output application/json
---
payload[?(($..count[0] >= "1") and ($..city contains "Pune"))] default []

Results will remain same with expression used with descendent and contains operator.

Happy Learning:)

Read More »

How to lookup CSV table using Dataweave2

 Hello Friends, 

In this article we will see how to lookup CSV file data without converting it to xml/json or any other message format.

Let's take one example where we need to lookup data from CSV file and retrieve/filter results from the dataset. Here, we will take one very common dataset with employee details information and then we will filter the data with some set of desired condition.

CSV table Input

Name	Location	Pincode
Niral	Pune	910002
Nikku	Pune	910005
Shruthi	Bhopal	843001
Manpreet	Chandigarh	890021
Little John	Mumbai	200011
Harry	Delhi	100011
Tom	Goa	500110

DW Script:

%dw 2.0
output application/json
---
(payload filter ((item, index) -> item.Location == "Pune"))

Output:

[
  {
    "Name": "Niral",
    "Location": "Pune",
    "Pincode": "910002"
  },
  {
    "Name": "Nikku",
    "Location": "Pune",
    "Pincode": "910005"
  }
]

In above example we are filtering data based on location. Here, we have not converted csv data in json before using filter. You can also store the results in dataweave vars and use those results based on your use-cases. 

Please find below some variations having same output as results:

%dw 2.0
output application/json
---
payload filter ($.Location == "Pune")

You can have multiple conditions also. Below expression, we are retrieving data from csv file inside project. You can use readUrl to load csv data from classpath and then use filter  based on Location and Pincode.

 

%dw 2.0
output application/json
var fiterResult= readUrl("classpath://csvFiles/ProfileDetails.csv","application/csv") filter ($.Location == "Pune" and $.Pincode == "910005")
---
fiterResult

Output:

[
  {
    "Name": "Nikku",
    "Location": "Pune",
    "Pincode": "910005"
  }
]

Happy Learning :)

Read More »

Validation module indepth - Mule4

 Introduction:

Mulesoft has provided a bunch of validation components which is very useful for message field validation during developments. These validations can be general or based on specific requirements. Validators are used to verify that parts of the Mule message/events to meet some kind of criteria which we specify.

If a message doesn't meet the validation criteria, validation component generates Validation error. Here, we can customize the exception message that is displayed in logs/responses.

Let's see some of important and frequently used validation components

mule4-validation-module

Is not null:

This components checks if the field value should be not null or else throws' error with "Name is Mandatory field"

<validation:is-not-null value="#[payload.name]" message="Name is Mandatory field"/>

Is email:

This components checks if the incoming email field is in valid format or not else throws validation error with message "Invalid Email format".

<validation:is-email email="#[payload.email]" message="Invalid Email format"/>

ALL vs ANY

To perform multiple validation at a time in mule flow. Mulesoft provides two scopes ALL and ANY which is used for these purpose.

ALL:- All validation should be passed from a set validation components defined in ALL scope.

<validation:all>
      <validation:is-not-null value="#[payload.name]" message="Name is Mandatory field"/>
      <validation:validate-size value="#[payload.name]" min="5" max="20" message="Name field size should be between 5 and 20"/>
</validation:all>

Here we are checking name field should not empty and name length should be minimum length 5 and max 20. It is similar to AND conditions like any programming language.

ANY:- Any of the validation components should be passed from a set of Validation components defined in ANY scope

<validation:any>
  <validation:is-not-null value="#[payload.name]" message="Name is Mandatory field"/>
  <validation:validate-size value="#[payload.name]" min="5" max="20" message="Name field size should be between 5 and 20"/>
<validation:any>

With Any scope we are ensuring any of the above condition should pass, it will pass the the validation. It is similar to OR conditions like any programming language.

Based on project requirement we can used these scopes with multiple validation components at a time.

Use of validation module Dataweave:

Now let us see how we can use these mule validation components can be used as Mule expression to make decisions without throwing any validation errors.

<choice>
   <when expression="#[ Validation::is-not-null(payload.name) and Validation::is-not-null(payload.name)]">
	<set-payload value='#[payload.name ++ " is a valid name."]'/>
   </when>
   <otherwise>
        <set-payload value='#[payload.name ++ " is NOT a valid name."]'/>
   </otherwise>
</choice>

Happy learning :)

Read More »

Pattern Matching in Dataweave Using Match Case

 Bonjour!

Just like any other programming language, In Mule4, dataweave provides match function to achieve the functionality of if-else statements. 

A match expression contains a list of case statements that can optionally have an else statement. Each case statement consists of a conditional selector expression that must evaluate to either true or false.

Let's consider a scenario where you need to update certain fields of incoming payload keeping the rest of the fields as it is, you can use a match and case. 

for example, the following is the input JSON object:

Input:

{

    "address": {

     "houseNumber" : null,

"street" : "IOC Road",

"pincode" : "111222",

"landmark" : "Statue of unity",

"city" : "Ahmedabad",

"state" : "Gujarat",

"country" : "India"

    }

}

Dataweave Transformation:

%dw 2.0

output application/json

var flag = if (payload.address.houseNumber == null) false else true

---

{

"updateAddressRequest":

payload.address mapObject (value,key,index) ->(

key as String match {

case "pincode" -> {(key): "000000"}

case "houseNumber" -> {(key): if(flag == false) "00" else ""}

else -> {(key):(value)}

}

)

}

Output:

{

"updateAddressRequest": {

"houseNumber": "00",

"street": "IOC Road",

"pincode": "000000",

"landmark": "Statue of unity",

"city": "Ahmedabad",

"state": "Gujarat",

"country": "India"

}

}

What we did here with the input JSON object is that we matched the keys using keyword match and then based on a condition updated the value of the key. We then used the else keyword in order to keep the rest of the fields the same as the input. 

In above example we are updating houseNumer and pincode field with case and other fields are remains same with else block.

This is also an example of how we can use the match case with mapObject. 

Pattern matching using Match Case can be done on Literal values, on expressions, on Data Types, and on Regular Expressions. 

Adios amigos!

Read More »

How to pass payload using Get method-Mule4

Hello Friends, 

I recently worked for one of API implementation where I needed to encrypt and send/receive one or more field in queryParms/headers while calling Get operation to external system.

As per API standard Get method should not pass payload as body while calling any Apis or 3rd party systems. Get method supports Uri Params, Query Params or Header Attributes.

In this article, I will show you the technique how you can pass encrypted payload using Http Get method. This is one of tricky interview question which normally asked in Mulesoft or Api Design related interviews.

As we know the when we encrypt any value it will not be a single string. Encrypted value may be of be multiple line as given below for PGP encryption for value 123456

-----BEGIN PGP MESSAGE-----

Version: BCPG v1.58

hIwDmCS94uDDx9kBA/9NIO0Kq9+yhXBLQveoykVjbECqXnGM8DargSsBxE2xBLQP

000W7mre5YpeSRJi4dvjVkAKPaxIFJhHlUNv38i5DkGD/z53SKtbKfYdfT+3oHT4

SVnIkcqWM+i+xawrKSk8Lz+j5GaJZ+wQrrxi8TP9RVW+wZlcbBNfxZVShDYp8dJX

AX9/x+1dFyO0EtHBipB0MLOqrb+sua028HGcg7E/sptn89JAgUcNEXOurVSfN6Ws

P/KxUMQY9+KKQQKavJ6LC92a6GJhVk4S+cnY1sxDLGtqKcGd+qCd

\u003dwGi2

-----END PGP MESSAGE-----

For both scenario(send/receive) we will be using (dw::core::Binaries) module from dataweave

toBase64(Binary): String

It takes Binary as input and returns result as base64 encoded String.

base64 Encoding

fromBase64(String): Binary

It takes base64 encoded String and returns original Binary data as a result.

base64 decoding

In this article we have seen how to convert encrypted payload to base64 encoded string and vice-versa. We can do the same conversion and pass with other payload type like JSON/XML etc..
These two dataweave functions are quite useful while dealing with payload with GET method.

Happy Learning :)

Read More »

Mulesoft Interview Questions With Answers - Part 2

Bonjour!

Let's take a look at some more Mulesoft basic interview-related questions!

Which component is used for schema validation and routing incoming Mule messages against RAML?

APIkit Router. 

Mule provides APIkit Router for Schema validation and routing the incoming Mule messages, serializing responses, validating payloads, Query parameters, URI parameters against RAML. 

Where are the HTTP, database configurations defined?

Global.xml

What is the purpose of a domain project?

The main purpose of a domain project is to have all the globally shared configurations for the projects. All the shared resources can be configured in a domain project. Mule applications can be associated only with one domain project. However, a domain project can be associated with multiple mule applications.

Does Cloudhub support Mule domain projects?

No, Mule domain projects neither supported in Cloudhub nor in Runtime Fabric. Domain projects are supported in on-premises environments.

Which connector is used for handling SOAP requests?

Web service Consumer.

How we can invoke REST or SOAP Services?

To invoke REST we can use HTTP and the Web service Consumer connector can be used to invoke SOAP services.

What is the use of Message Enricher?

In Mule 3, Message enricher is a component used to enrich the current message with additional information from a separate resource without disturbing the current payload.

Enricher is used in scenarios where the target system needs more information than the source system can provide. It calls an external system or doing some transformation to the current payload and saving it in a target variable.

Do we have Message Enricher in Mule 4? 

In Mule 4, We don't have a message enricher anymore. 

Now, every component is embedded with a target variable which solves the purpose. 

 

How do you override application properties?

You can configure your Mule application to automatically load properties from properties files based on different environments (Production, development, testing, QA) using property placeholders. 

1.  Create a properties file for each development environment in your Mule application.
2. Configure a property placeholder in your application to look for the environment at the time of launch.
3. Set environment variable one each for the different environment.

The property placeholder will then take up an environment variable value and based on that use the properties file.

How can we store sensitive data such as Client ID and Client Secret? 

Properties in Mule 4 can be encrypted to store sensitive data such as Client ID and Client Secret. We can encrypt a .yaml or .properties file using the secure properties module. For more details about the use of  secure properties: securing-configuration-properties-in-mule

How to install anypoint security suite

how-to-use-secure-properties-in-dw2

Give some examples of popular Mulesoft connectors? 

HTTP, File, FTP, SFTP, Database, Salesforce, SAP, AWS, etc.

Can you create a custom connector? 

Yes. 

When an API specification is published to Anypoint Exchange, it automatically generates a connector for it. You can find out a list of Mulesoft connectors here.

Advantages of Mulesoft Anypoint connectors? 

• It provides integration of Mulesoft application with different 3rd party systems like Facebook, Twitter, Salesforce, etc.
• It is easier to maintain and no need to spend time on optimization and security.
• It provides reusability which helps in speed-up development.

Is Mulesoft PaaS or SaaS? 

Mulesoft provides iPaaS (Integration Platform as a Service). It provides connectivity across on-premises and SaaS applications.

In Mule4 how you can generate custom errors.

By using the Raise Error component

In Mule 4, how errors are handled in a mule flow?

When an error is thrown in Mule flow while processing an event first step is it stops normal flow execution and it will not go to the next components. Then depending on the error type either error will be caught and handled inside the error handling block or thrown up to its parent flow.

Where we can add an error handler in the Mule 4 application?

An error handler can only be added to

– A flow

- A private flow

– A Try scope

– An global error handler (outside of any flows) is then referenced to any of the error handling strategies.

Note: Error handler can not be added to subflows.

What is the Mule default error handler?

When there is no error handler defined for any flow or by using try scope, the Mule default error handler will be used. This default handler should be configured globally.

What if the default error handler not configured in the Mule application?

If the default error handler is not configured then Mule runtime will provide its own default behavior

– First, it stops the execution of the flow and logs information about the error.

– Implicitly and globally handles all errors thrown in Mule applications.

– It automatically rethrows the error.

– Most important we cannot configure it.

What are error scopes in mule4 to handle errors?

– On Error Continue scope

– On Error Propagate scope

 

Can you please explain the difference between these two error scopes in Mule4?

What is the use of Try scope in Mule4?

Try Scope is very useful when you need to group a set of events/ components and want to handle errors based on your specific requirements. It acts as a private flow inside the parent flow. For Unhandled errors, it propagates to its parent flow.

Read More »

RAML 1.0 Multiple Types for a Single Request

 Bonjour!

You must have come across a situation where you need to define two different Types for a particular resource in your RAML specification.

In this article, I'll show you how to do that!

you can define multiple Types using the pipe symbol ("|") as follows: 

The request Types can be defined in the header of the RAML. The examples can be specified using facet "examples" and added in the form of either files using the keyword "include" or the example itself.

 

Adios amigos!

Read More »

Mulesoft Interview Questions With Answers - Part 1

Bonjour!

What is Mulesoft?

The first and the most basic question asked in Mulesoft developer interviews. 

Mulesoft is the vendor that provides an integration platform to help businesses connect data, applications, and devices.

Mule is the runtime engine of the Anypoint Platform. It is a lightweight Java-based enterprise service bus (ESB) that allows developers to connect applications and exchange data. It can connect any systems by using different protocols like HTTP, JMS, JDBC etc.

What is Anypoint Platform?

Anypoint platform is a hybrid integration platform which is highly productive, unified and helps to create a smooth distributed systems. It helps to manage full API lifecycles. It provides unified platform to deploy, manage, monitor and rapid development of apps into single place.

What is a Mule Application?

A Mulesoft application is a package of codes and files that runs inside Mule Runtime. Mulesoft application can be triggered with set of events that can be external and inside mulesoft. It can process events and route to next components or endpoints. Endpoints can be external system, another mulesoft application or local resources like file.

A Mule Application may consists of one or more flows, subflows and private flows.

What is the difference between Flow, Subflow, and Private flow?

Flow is a message/event processing block. It can have Mule sources (e.g. an HTTP listener) to trigger the execution of a flow. It can have its own processing strategy and error handling strategy.

Subflow is a scope that enables you to process messages synchronously. It doesn't have any source or error handling block. It inherits the processing strategy and error handling strategy from parent flow. It can be used to split common logic and reusability.

Private flow is a flow without a Mule source. It can be synchronous or asynchronous based on the processing strategy selected. They can have their own error handling strategy.

All three above can be called using a flow-ref. Source will be skipped while calling using flow-ref.

How Mule Applications can be build?

Mule application can be created using online flow-designer and offline by downloading Anypointsudio. 

Link for Online flow-designer

Link to download AnypointStudio

How Mule4 Application flow executes?

Mule4 application flows executes events in multiple threads. Mule4 automatically optimizes performance and uses thread switching and parallel processing if required.

Variables in Mule 4 vs Mule 3?

In Mule 3, we had three types of variables: Session variables, Record variables, and Flow variables.

In Mule 4, we only have Flow variables. These variables can be accessed using the keyword "vars".

Example:

vars.variableName

What is API-led connectivity?

API-led connectivity is a methodical way to connect applications to data through reusable APIs. APIs are created in three distinct layers: Experience, Process, and System.

API Life Cycle?

An API life cycle consists of 3 layers:

The first step towards API development is API Specification. This phase involves designing of the API based on the requirements of business. An API Specification is a blueprint of an API.

Anypoint Platform provides Design center to design API's REST interfaces.

The next phase is API Implementation. This phase involves actual implementation of the API.

Anypoint Platform provides Flow Designer which is a web based tool to create a prototype of the API.

Anypoint Studio is the IDE for development and testing of Mule applications. 

Final phase in API life cycle is API management, uses a tool named API manager on Anypoint Platform which is used to apply policies to secure APIs and restrict access. 

What are the advantages of API-led approach?

• Mulesoft prefers Design first approach for API development that helps non-technical stakeholders easy to understand the API specs/contracts.
• It simplifies the interface between Client and application by hiding complex details.
• Business validation and contracts can be defined at API specification. So, developers don't need to invest time to write code for validation and error type checks.
• It encourages "fail fast" approach which helps in fast and parallel development.
• It encourages reusability of assets and collaboration across teams.
• Apis can be mocked before implementation starts which helps early feedback and parallel development.
• Mulesoft anypoint platform provides tools to write, publish, share and manage versions of API.

What is RAML?

RAML stands for RESTful (REpresentational State Transfer) API Modeling Language. It's a way of describing a RESTful API that is readable by both human and computers. 

They use https methods such as GET, PUT, POST, DELETE, etc.

Difference between URI parameters and Query parameters.

URI parameter is used to specify a specific resource whereas Query parameter is used to filter or sort those resources.

let's say we need to filter products based on productType, we use query parameters in this case:

GET: /products?productType=electronics

Let's consider another scenario where we need to retrieve particular product details, we use URI parameters in this case:

GET: /products/{productID}

here, productID will be replaced by a value.

Error Handling in Mule 4?

Error handling in Mule 4 is redesigned. It has three mechanisms to handle errors now.

1. On-Error Propagate:  In an On-error Propagate scope, when an error occurs, it terminates the current execution of flow and then throws the error to its parent flow where it's handled.

2. On-Error Continue: In case of an error in an On-error Continue scope, the execution of the flow continues as if successful sending a 200 OK response.

3. Try Catch scope: A Try Catch scope is used to handle errors on individual components. It is especially useful when you need to separate error handling strategies for different components in a flow.

What are the Logger levels?

There are 5 logger levels namely: DEBUG, ERROR, INFO, TRACE, and WARN. 

What does scatter-gather return?

A scatter-gather returns a Mule event.

What is the use of Async scope?

Async scope can be used to trigger flows subflows or any components asynchronously from a parent flow. Async scope is very helpful while using logger component or parallel processing requires.

What is the difference between map and mapObject?

map operator is used for array. It takes an array as input and returns an array.

mapObject operator is used for object with key: value pairs. It takes an object as input and returns an object.

map invokes lambda with two parameters: value and index from the input array. 

Example:

%dw 2.0
output application/json
---
["apple", "orange", "banana"] map (value, index) -> { 
    (index) : value}

Output:

[ { "0": "apple" }, { "1": "orange" }, { "2": "banana" } ]

mapObject invokes lambda with key, value, or index for mapping given object.

Example:

%dw 2.0
output application/json
---
{"chandler":"bing","monica":"geller"} mapObject (value,key,index) -> { 
    (index) : { (value):key} }

Output:

{ "0": { "bing": "chandler" }, "1": { "geller": "monica" } }

What is batch processing?

Mule allows you to handle large quantities of data in batches. 

This is achieved using a Batch job scope which splits messages into individual records, performs actions on records, and reports the results. A Batch job scope processes data asynchronously.

Each batch job contains three different Processing Phases:  

1. Load and Dispatch: In this phase, Mule splits the message, creates a Batch Job Instance, a persistent queue and associates it with the Batch Job Instance.

2. Process: This phase does the actual processing of records asynchronously. 

3. On Complete: This is an optional phase where you can configure the runtime to create a report or summary of records. 

Stay tuned for more!

Read More »