Posts

Parallel Processing of Kafka events with order in MuleSoft

Image
As we know Kafka is a streaming platform, It best suits in one of the scenarios where same event has to be feed across different instances. This can be achieved by creating different consumer group Ids.  Consider a scenario, there is a Kafka topic with four partitions and a running application is consuming the messages actively from the topic. To provide the best performance, the order of consuming messages from the different partitions are handled by Kafka internally. Now say you are expecting more load on specific window where you will be expected to enable horizontal autoscaling for your api. Also, your api is maintaining order on the events to deal with CURD operations. 🤯 How it can be achieved? 🤔🤔 You can't travel on multiple boats at the same time 😝but Kafka can handle processing of multiple events at the same time😅. This article provides the understanding on how to handle CURD operation events in parallel processing with Kafka as streaming platform. 👀 Exciting isn'...

MuleSoft Integration With Seek Kafka Connector

Image
We know MuleSoft provides various option to integrate with different platforms like salesforce, database, AWS, Azure and one among available options is Kafka too. When I started my journey of integrating Kafka in MuleSoft using a seek connector, it was hard to find any relative information on the internet for reference😕. Hence here I am summarizing the possible information that might help. By the way for beginners, Kafka is streaming platform, it can be used to consume same events across different application with unique consumer group Id.  So, letsssss start to understand and fly like a Dragon........💃 We have following connectors provided by MuleSoft to integrate with Kafka.  Publish Consume Message listener Batch message listener  Commit The Star of this blog SEEK  In this blog the focus of light is on seek.  For the seek connector you can define the offset and partition that you want to start consuming from, irrespective of the event state if that was cons...