Anypoint MQ Circuit Breaker - Mule4

In integration development there are lot of patterns that we can follow based on project requirements. Circuit Breaker is one of them.
In this article we will see how Circuit Breaker can be used and what we should consider while implementing it. 

Circuit breaker design pattern is used when:

• To handle error scenarios and message reprocessing strategy.

Mulesoft has provided circuit breaker capability with Anypoint MQ component.
You can add dependency inside your pom as below:

<dependency>
<groupId>com.mulesoft.connectors</groupId>
<artifactId>anypoint-mq-connector</artifactId>
<version>3.1.0</version>
<classifier>mule-plugin</classifier>
</dependency>

Now, I will show you how you can design and configure your mule flows for implementing Circuit Breaker capability.


Below is the sample circuit breaker mule flow design:

We will explain each and every part the above flow design below:

Circuit Breaker Configuration:

Go to global elements section create a Circuit breaker configuration as below:

This configuration can be shared across multiple Subscriber resources in the project.

On Error Types – Type of Errors based on which you want to Open the Circuit.
Errors Threshold – No. of Failures in Succession – Ex: 5
Trip Timeout – Duration to keep the Circuit Open once the ErrorThreshold is reached – Ex: 60 Minutes

Subscriber Configuration

Anypoint MQ Subscriber Source provides Circuit Breaking Capability.
                                                       

 
1. Select the Anypoint MQ Subscriber Source.
2. Click on the Advanced tab.
3. Provide the Circuit Breaker from Global Reference.

AMQ Message Processing\Listening Strategy:

This section describes the methodology how we are going to follow to process messages from Anypoint MQ. This process helps in effectively handling reprocessing and retrying scenarios and also simplifies the process from a maintenance perspective.


Steps:

1. Configure the Subscribe processor
2. Subscriber Type – Prefetch
3. Max Local Messages –  default it to 5, you can change based on requirement for further tuning.
4. Acknowledgment Mode – Set to MANUAL, As the Acknowledgement Mode is set to MANUAL, we need to Ack the message post successful completion of the flow. So that the message doesn’t get retried/consumed.

Circuit Breaker and AMQ Listening Strategy both need to be configured in our components where we want messages to be reprocessed and retried in a configurable manner.

In case of Error Scenarios,

Based on your requirement you can catch the error and subsequent to Error Handling, Nack the message so that the message stays on the queue and will be subsequently redelivered.

You can use Ack the message so that the message is removed from the queue in case of success scenarios.

This ackToken you can store in variable which can be used for Ack/Nack of the message.

MQ messages stores in queue after getting Nack

Please find sample Mule project in Github Error-Handling-using-Anypoint-MQ-Circuit-Breaker

Read More »

Queueable Apex - The middle way

The last post was a comparison between batch apex and @future apex. There are some advantage and disadvantage with both of them. When batch and @future needs to meet in between - queueable comes for rescue.

There are a few things that @future did not provide:

1. @future requires the arguments be primitives, which means reconstructing a structure once the method is called.
2. Difficult access to job ID. The executeBatch method returns a jobID, while calling an @future job does not give you the ID of the related job.
3. No chaining - Chaining of batch apex is not allowed.

Q - Why we go for queueable Apex?

A - When the job such as extensive database operations or external Web service callouts is long running and is running asynchronously and you want to monitor the job status. It can be done via queueable apex and adding a job to the ApexJob queue. In this way, your asynchronous Apex job runs in the background in its own thread and doesn’t delay the execution of your main Apex logic.

Queueable jobs are similar to future methods in that they’re both queued for execution, but they provide you with these additional benefits.

Getting an ID for your job: When you submit your job by invoking the System.enqueueJob method, the method returns the ID of the new job. This ID corresponds to the ID of the AsyncApexJob record.
This ID helps you identify your job and monitor its progress, either through the Salesforce user interface in the Apex Jobs page, or programmatically by querying your record from AsyncApexJob.

Using non-primitive types: Your queueable class can contain member variables of non-primitive data types, such as sObjects or custom Apex types.

Chaining Jobs: In queueable you can also chain Jobs from a running job. It is very helpful if some processing needs to be done which depends on the last results.

Example:
This example is an implementation of the Queueable interface. The execute method in this example inserts a new account.

public class QueueableExample implements Queueable {

    public void execute(QueueableContext context) {
        Account acc = new Account(Name='Test Account Neeraj');
        Insert acc;     
    }
}

To add this class as a job on the queue, call this method:
ID jobID = System.enqueueJob(new QueueableExample());

After you submit your queueable class for execution, the job is added to the queue and will be processed when system resources become available. We can monitor the status of our job programmatically by querying AsyncApexJob or through the user interface Setup --> Monitor --> Jobs --> Apex Jobs.

Test class for queueable apex:

@isTest
public class AsyncExecutionExampleTest {
    static testmethod void testAccount() {
     
        Test.startTest();     
        System.enqueueJob(new QueueableExample());
        Test.stopTest();

        Account acct = [SELECT Name FROM Account WHERE Name='Test Account Neeraj' LIMIT 1];
        System.assertEquals('Test Account Neeraj', acct.Name);
    }
}

Few things can be noted here:

• The execution of a queued job counts against the shared limit for asynchronous Apex method executions.
• We can add up to 50 jobs to the queue with System.enqueueJob in a single transaction.
• Use Limits.getQueueableJobs() to check how many queueable jobs have been added in one transaction.
• No limit is enforced on the depth of chained jobs.
• We can add only one job from an executing job with System.enqueueJob, that means only child job can exist for parent queueable job.

Read More »

Access Anypoint Platform in Anypoint Studio

Issue:

You can not access Anypoint Platform after configuring External Identity in Anypoint Studio.

Reason:

One of most common issues is the wrong Organization Domain configuration.

HowTo:

Please follow below steps to validate if the Organization domain is properly configured:

Open Anypoint Studio
Go to Window -->Preferences --> Anypoint Studio --> Authentication --> Add Button

 
Now click on Configure --> Select External Identity

Provide your valid Organisation domain --> Select OK
Please validate your Organization domain field. The value must be your valid Organization domain.

Once validated you can login your Anypoint Account with your user-id and password. You also can configure expires days.

Finally select Apply and Close button.


Happy Learning :)

Read More »

Configure Git plugin in Anypoint Studio

Objective

How to install Git plugin in Anypoint studio. 
The installation advantage of git plugin to Anypoint studio that you can do following things through AnypointStudio:

• You can clone a existing Git repository.
• Checkin your code to git repositories
• Create a git repositories and many more...

Steps to Install Git Plugin to AnypointStudio

1. Go to http://wiki.eclipse.org/EGit/FAQ#Where_can_I_find_older_releases_of_EGit.3F,  and copy the "p2 repository URL"  for the Eclipse version you are using (or select the closest version to what you have installed)

2. In Studio, click the Help menu --> select Install New Software…

3. In the Work with field of the Available Software panel, click on Add button, and add a New name and the URL copied. Click ok

4. Select Git integration for Eclipse, and continue installation as normal.

After Installation you will need to restart your AnypointStudio and then you will find your Git plugin perspective. You can get from the quick search option as below:

or from Menu Bar, 
Window--> Perspective--> Open Perspective --> Other--> Select Git:

Happy Learning :)

Read More »

Munit coverage for APIKit flows Part2 - Mule4

In my previous articles How to generate Mule flow using RAML and MUnit complete coverage for ApiKit flows Part1 - Mule4 we have seen how Mule application can be generated using APIkit RAML and how we can test our routing flows using MUnit Test Suites.

Now here in this article we will see how to write MUnit test cases for flows that were created with APIkit RAML for APIKit Validation scenarios.

APIkit Router validates the incoming requests, HTTP methods, URI and URI parameter, header and query parameters against the structure or validation defined in RAML. It also routes the message to the respective flow and serials response. Message routing consists of incoming API requests to existing Mule flows and returning outgoing HTTP response codes to exceptions.

Things to consider while writing MUnits for Validation flows:

Above configuration is needed to validate HTTP response code. This you can configure as per your test scenarios.

After adding MUnit for APIKit Validation flows your MUnit coverage report has been increased as shown below:

Please find sample Mule project in Github munit-for-apikit-flow-part2-mule4

Happy Learning :)

Read More »

Comparing JSON string payload to another JSON string - Mule4

In REST API Mule development in which system returns a JSON response, it is always advisable to write MUnit tests and assert the entire JSON response payload to avoid contract conflicts/mismatches.

Objective:

Our objective is loading expected JSON data from a file and comparing it with actual MUnit payload which we got from the tested flow.

Problem Description:

If You are using Mule 4 and seeing MUnit test fails comparing JSON string payload to another JSON string.
The application produces an output similar to below:
%dw 2.0
output application/json
---
{
  message: "Success PUT"
}

The sample payload below to compare for the MUnit test is saved in a file:
{"message":"Success PUT"}

After running your MUnit you may get similar error like below:

java.lang.AssertionError: The response payload is not correct! at file: [munit-for-apikit-flow-part2-mule4-apikit-test.xml], line: [22]
Expected: "{\"message\":\"Success PUT\"}" as String {class: "java.lang.String"}
     but: "{\n  \"message\": \"Success PUT\"\n}" as String {encoding: "UTF-8", mediaType: "application/java; charset=UTF-8", mimeType: "application/java", class: "java.lang.String", contentLength: 30} at (root)

at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)

Problem Solution:

Use "readUrl" function to compare both types as JSON with JSON:

<munit-tools:assert-that expression="#[%dw 2.0 output application/json ---payload]" is='#[MunitTools::equalTo(readUrl("classpath://scaffolder/response/put_201_apikit-put_application_json.json", "application/json"))]' message="The response payload is not correct!" doc:name="Assert That - Payload is Expected" doc:id="bb6653e2-3022-4644-9099-204d13a5fce6"/>
   

Please find sample Mule project in Github munit-for-apikit-flow-part1-mule4

Happy Learning :)

Read More »

MUnit coverage for APIKit flows Part1 - Mule4

In my last article How to generate Mule flow using RAML.
Now here in this article we will see how to write MUnit test cases for flows that were created with APIkit RAML.

Application takes a RAML file and maps it to an implementation of an API in Mule.

We will see the implementation routes that handles incoming request based on Http methods (GET, POST, PUT, DELETE) and URL using MUnit.

Now let's start writing MUnits for routing flows:

Step 1: Generate Munit Flow from Apikit Flow: 

Right click on APIKit router-->Create Test Suite for Munit

Step 2: Run MUnit Suite:

Right click Munit flow--> select Run MUnit suite

Step 3: Flow execution sequence

Please check your MUnit test configuration if your MUnit is not behaving properly. Enable flow sources must be configured to List of Flows section as shown below:

Then MUnit will find the source and will execute the test accordingly.

Step 4: Coverage report:

Please find sample Mule project in Github munit-for-apikit-flow-part1-mule4

After running MUnit you will get MUnit coverage report like shown above. But If you have noticed APIKit main flow is having very less coverage percentage and it only covers resource flows. So, how we can increase or get the MUnit coverage for APIKit main flow which includes exception handling processors?

For this you needed to write MUnit additionally to cover these scenario which I will explain in my next article.

Happy Learning :)

Read More »

Create APIKit project using raml-Mule4

In this article we will see how we can create or generate Mule 4 project using RAML.
We will also see how Mule flow responds in case of Success and Validation scenarios.

RAML stands for RESTful API Modeling Language

RAML (Rest API Modeling Language) is based on YAML format, which is used to design REST APIs. RAML provides various features, including standardization, re-usability, easy readability, and much more. RAML provides a structure to the API which is useful for developers to start development process and it also helps clients to understands the request and response structure outcomes before hand.

What RAML contains?

• Endpoint URL with its Query parameters and URI parameters,
• HTTP methods to which API is listening to (GET, POST, PUT, DELETE),
• Request and response schema and sample message,
• HTTP response code that an API will return (eg: 200, 400, 404, 415, 500).

Lets jump into code build:

Step 1: Download a sample RAML file link

Step 2: Create Mule project: From AnypointStudio

             GoTo File--> New --> Mule Project

• Provide Project Name
• Select Runtime
• Import RAML file by specifying file location or URL
• Click finish button

Step 3: Building and Running Mule Project

Right click on the project --> Run As --> Mule Application

Step 4: Test your deployed App through API Console

Step 5: Check Success Scenario

Flow execution sequence in Success Scenario

In API console, When we click on "GET"

• Request will be sent to HTTP listener endpoint.
• HTTP listener takes the request and forwards to APIKit router
• APIKit validates the incoming requests and passes it to designated flow.
• Here it is passing to "get:\apikit-login:apikit-raml-flow-Mule4-config" flow
• Request is processed and gives a 200 OK response.

Step 6: Check Error Scenario

Flow execution sequence in Error Scenario(Bad Request)
On Api Console, When we execute DELETE operation.

• Enter incorrect value in Delete Id.
• Request will receive to HTTP endpoint and forwards to APIKit router.
• APIKit router validates the incoming request.
• On validation failed APIKit throws an error and which is caught by error handler.
• Error handler passes the flow execution to APIKIT:BAD_REQUEST scope and gives response with  status code 400

Below are some error codes and their meanings which we normally uses in our development.

Please find sample Mule project in Github raml-apikit-flow-mule4

Next article we will see how to write MUnit test cases for these APIKit flows - Part 1

Read More »

Running MUnit for single file flow and Tags-Mule4

In this Article we will see how to run single munit at a time through command line and from Anypoint studio.

Also we will see how to run MUnits using Tags from command line

Execute from Command Line

Using "mvn test" at the terminal will test all MUnit tests, but here we describe how to test only one MUnit file

Run the following command at the project root folder.

In the following example, the MUnit test file is called run-single-munit-test-suite.xml

mvn clean test -Dmunit.test=run-single-munit-test-suite.xml

Running Single MUnit test case

XML file contains several tests, you may add the test name like this:

mvn clean test -Dmunit.test=abc.xml#testName

mvn clean test -Dmunit.test=mvn clean test -Dmunit.test=run-single-munit-test-suite.xml#run-single-munit-test-suite-run-single-munitFlowTest
  
In this case, the "testName" is the name of the test to be executed.

Execute from AnypointStudio

Right click in Munit file --> Run MUnit suite

Right click on test Name--> Run MUnit test

Running Munit using Tags CommandLine

Tags are important to categorize the MUnits that needs to be executed based on the requirement. It is more like developer friendly to execute the different test scenarios.

mvn clean test -Dmunit.tags=flow

Above command will run all the munit tests tagged as name "flow"

You can also use combination as below:

mvn clean test -Dmunit.test=run-single-munit-test-suite.xml -Dmunit.tags=flow

Please find sample Mule project in Github run-single-munit-Mule4

Happy learning :)

Read More »

Munit Issue with Async block inside subflow Mule4 - Resolved

In this article We will see the issue related while running Munit with Async block inside sub-flows. We will also see why this error normally occurs while running Munit and what are the options to resolve it.

java.lang.IllegalStateException: Couldn't register an instance of a MessageProcessorChain
at org.mule.runtime.config.internal.LazyMuleArtifactContext.lambda$createBeans$20(LazyMuleArtifactContext.java:399)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at org.mule.runtime.config.internal.LazyMuleArtifactContext.createBeans(LazyMuleArtifactContext.java:386)

Caused by: org.mule.runtime.core.privileged.registry.RegistrationException: Could not register object for key mule-munit-async-subflow-issueSub_Flow@729342238

Caused by: org.mule.runtime.api.lifecycle.LifecycleException: Failed to invoke lifecycle phase "start" on object: SubFlowMessageProcessorChain 'mule-munit-async-subflow-issueSub_Flow' 

[ 

  com.mulesoft.mule.runtime.core.internal.processor.TransformMessageProcessor@534ae04a, 

  org.mule.runtime.core.internal.processor.AsyncDelegateMessageProcessor@41cc2f00

]

Caused by: java.lang.NullPointerException

at org.mule.runtime.core.internal.component.ComponentUtils.getFromAnnotatedObject(ComponentUtils.java:37)

Problem Analysis:

As we know sub-flow is synchronous and single threaded in nature. Also Async block every time creates a new thread from running thread to complete the tasks in Async mode. So, in sub-flow scope it should never allow to create a separate thread from the running thread.

Resolution:

Method I: Change sub-flow to private flow
By changing the sub-flow to private flow, you can resolves this issue. As flow supports multi thread execution. So when Async block will called a new thread will be created and registered to its main thread and finishes its execution in Async mode.

Method II: Add Try scope on Async scope

Second method is to add Try block on top of Async block to make the execution transactional or running in single thread. So when Munit starts running it will execute and finish its tasks without any issue. 

 

Which one to choose?

It all depends on your requirement and design. Normally adding try block in sub-flows is a better option if you don't want to change your existing design.


Happy Learning :)

Read More »