Partitioning In Spring Batch

Introduction

As in a batch job huge no of records are processed, it faces performance bottlenecks sooner than later. Different performance bottlenecks that can be cited during batch job execution are long execution period, OutOfMemoryError, bad records which abort the job immediately etc. So while writing a real-time batch job, we have to be utmost careful and take these parameters into consideration. That will help us scale up the batch job properly.

Though there are various ways to scale up a batch job, here I shall show you how to scale up a Spring Batch job using local partitioning. Partitioning is a mechanism of dividing a step into multiple threads where each thread executes a chunk of data in parallel. Here original step is called Master while steps that execute through threads are called Slaves. Some programmers refer to it as Master-Slave approach. Again partitioning can be of two types, local and remote. Remote partitioning is beyond the scope of today's discussion.

Springing Into Batch Job

Spring Batch

A batch job is a software job that processes large volume of data or records in a short span of time without any human intervention. As such batch job plays a crucial role in the functioning of many big enterprise applications. Batch job is there in the software industry since the beginning and I am sure it will be there till the end. Some examples of batch jobs are indexing files, updating the inventory catalog for an online store etc.

Spring Batch is a module of popular Spring Framework which enables writing batch applications / jobs easily and in an effective manner. Before Spring Batch hit the market in the year 2007, industry heavyweights used to write batch applications using their own proprietary technologies and legacy frameworks. Apart from defining the standard, Spring Batch provided some ready-to-use components and all the advantages of Spring framework like POJO, dependency injections etc. to write complex batch applications.