By kswaughs | Tuesday, March 22, 2016

Spring Boot Batch Job + Scheduler Example

We have seen how to schedule a task using spring boot and how to run a batch job using spring boot in below posts.

In this example, We will see how to run that Batch job with spring scheduler using spring boot.

Step 1 : By default, the batch job runs immediately when we start the application. So we have to disable the auto run feature in application.properties file.

spring.batch.job.enabled=false

Step 2 : Configure JobLauncher in one more configuration class with defining required dependant components.

BatchScheduler
package com.kswaughs.config;

import org.springframework.batch.core.launch.support.SimpleJobLauncher;
import org.springframework.batch.core.repository.JobRepository;
import org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean;
import org.springframework.batch.support.transaction.ResourcelessTransactionManager;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.annotation.EnableScheduling;

@Configuration
@EnableScheduling
public class BatchScheduler {

    @Bean
    public ResourcelessTransactionManager transactionManager() {
        return new ResourcelessTransactionManager();
    }

    @Bean
    public MapJobRepositoryFactoryBean mapJobRepositoryFactory(
            ResourcelessTransactionManager txManager) throws Exception {
        
        MapJobRepositoryFactoryBean factory = new 
                MapJobRepositoryFactoryBean(txManager);
        
        factory.afterPropertiesSet();
        
        return factory;
    }

    @Bean
    public JobRepository jobRepository(
            MapJobRepositoryFactoryBean factory) throws Exception {
        return factory.getObject();
    }

    @Bean
    public SimpleJobLauncher jobLauncher(JobRepository jobRepository) {
        SimpleJobLauncher launcher = new SimpleJobLauncher();
        launcher.setJobRepository(jobRepository);
        return launcher;
    }

}

Step 3 : Follow the below methods

Import the above Configuration class to your batch configuration class.

Get the reference of the configured JobLauncher through autowired injection.

Write a new method annotated with @Scheduled and desired cron expression.

Run the JobLauncher with passing job bean and custom job parameter.

BatchConfiguration
package com.kswaughs.config;

import java.util.Date;

import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobExecutionListener;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.support.RunIdIncrementer;
import org.springframework.batch.core.launch.support.SimpleJobLauncher;
import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.item.file.FlatFileItemReader;
import org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper;
import org.springframework.batch.item.file.mapping.DefaultLineMapper;
import org.springframework.batch.item.file.transform.DelimitedLineTokenizer;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Import;
import org.springframework.core.io.ClassPathResource;
import org.springframework.scheduling.annotation.Scheduled;

import com.kswaughs.beans.Order;
import com.kswaughs.beans.SvcReq;
import com.kswaughs.order.OrderSvcInvoker;
import com.kswaughs.processor.JobCompletionNotificationListener;
import com.kswaughs.processor.OrderItemProcessor;

@Configuration
@EnableBatchProcessing
@Import({BatchScheduler.class})
public class BatchConfiguration {

    @Autowired
    private SimpleJobLauncher jobLauncher;

    @Autowired
    public JobBuilderFactory jobBuilderFactory;

    @Autowired
    public StepBuilderFactory stepBuilderFactory;

    @Scheduled(cron = "1 53/3 17 * * ?")
    public void perform() throws Exception {

        System.out.println("Job Started at :" + new Date());

        JobParameters param = new JobParametersBuilder().addString("JobID",
                String.valueOf(System.currentTimeMillis())).toJobParameters();

        JobExecution execution = jobLauncher.run(processOrderJob(), param);

        System.out.println("Job finished with status :" + execution.getStatus());
    }

    @Bean
    public Job processOrderJob() {
        return jobBuilderFactory.get("processOrderJob")
                .incrementer(new RunIdIncrementer())
                .listener(listener())
                .flow(orderStep()).end().build();
    }

    @Bean
    public Step orderStep() {
        return stepBuilderFactory.get("orderStep").<Order, SvcReq> chunk(3)
                .reader(reader()).processor(processor()).writer(writer())
                .build();
    }

    @Bean
    public FlatFileItemReader<Order> reader() {
        FlatFileItemReader<Order> reader = new FlatFileItemReader<Order>();
        reader.setResource(new ClassPathResource("PhoneData.csv"));
        reader.setLineMapper(new DefaultLineMapper<Order>() {
            {
                setLineTokenizer(new DelimitedLineTokenizer() {
                    {
                        setNames(new String[] { "orderID", "orderName" });
                    }
                });
                setFieldSetMapper(new BeanWrapperFieldSetMapper<Order>() {
                    {
                        setTargetType(Order.class);
                    }
                });
            }
        });
        return reader;
    }

    @Bean
    public OrderItemProcessor processor() {
        return new OrderItemProcessor();
    }

    @Bean
    public ItemWriter<SvcReq> writer() {
        return new OrderSvcInvoker();
    }

    @Bean
    public JobExecutionListener listener() {
        return new JobCompletionNotificationListener();
    }

}

In this example, I configured the job to start at '5 PM 53 minutes 1 second' and run for every 3 minutes till 6 PM with cron expression.

Output console logs

Job Started at :Tue Mar 22 17:53:01 IST 2016
2016-03-22 17:53:01.052  INFO 10932 --- [pool-2-thread-1] o.s.b.c.l.support.SimpleJobLauncher: Job: [FlowJob: [name=processOrderJob]] launched with the following parameters: [{JobID=1458649381004}]
2016-03-22 17:53:01.078  INFO 10932 --- [pool-2-thread-1] o.s.batch.core.job.SimpleStepHandler: Executing step: [orderStep]
Converting (Order [orderID=101, orderName=Apple IPhone]) into (SvcReq [id=101, name=APPLE IPHONE])
Converting (Order [orderID=102, orderName=Samsung Galaxy Y]) into (SvcReq [id=102, name=SAMSUNG GALAXY Y])
Converting (Order [orderID=103, orderName=Moto E]) into (SvcReq [id=103, name=MOTO E])
calling web service:SvcResp [id=101, message=APPLE IPHONE Processed successfully]
calling web service:SvcResp [id=102, message=SAMSUNG GALAXY Y Processed successfully]
calling web service:SvcResp [id=103, message=MOTO E Processed successfully]
Processed items:3
Converting (Order [orderID=104, orderName=Moto X]) into (SvcReq [id=104, name=MOTO X])
Converting (Order [orderID=105, orderName=Yuphoria]) into (SvcReq [id=105, name=YUPHORIA])
calling web service:SvcResp [id=104, message=MOTO X Processed successfully]
calling web service:SvcResp [id=105, message=YUPHORIA Processed successfully]
Processed items:2
BATCH JOB FINISHED SUCCESSFULLY
2016-03-22 17:53:01.350  INFO 10932 --- [pool-2-thread-1] o.s.b.c.l.support.SimpleJobLauncher: Job: [FlowJob: [name=processOrderJob]] completed with the following parameters: [{JobID=1458649381004}] and the following status: [COMPLETED]
Job finished with status :COMPLETED

Job Started at :Tue Mar 22 17:56:00 IST 2016
2016-03-22 17:56:01.006  INFO 10932 --- [pool-2-thread-1] o.s.b.c.l.support.SimpleJobLauncher: Job: [FlowJob: [name=processOrderJob]] launched with the following parameters: [{JobID=1458649560996}]
2016-03-22 17:56:01.031  INFO 10932 --- [pool-2-thread-1] o.s.batch.core.job.SimpleStepHandler: Executing step: [orderStep]
Converting (Order [orderID=101, orderName=Apple IPhone]) into (SvcReq [id=101, name=APPLE IPHONE])
Converting (Order [orderID=102, orderName=Samsung Galaxy Y]) into (SvcReq [id=102, name=SAMSUNG GALAXY Y])
Converting (Order [orderID=103, orderName=Moto E]) into (SvcReq [id=103, name=MOTO E])
calling web service:SvcResp [id=101, message=APPLE IPHONE Processed successfully]
calling web service:SvcResp [id=102, message=SAMSUNG GALAXY Y Processed successfully]
calling web service:SvcResp [id=103, message=MOTO E Processed successfully]
Processed items:3
Converting (Order [orderID=104, orderName=Moto X]) into (SvcReq [id=104, name=MOTO X])
Converting (Order [orderID=105, orderName=Yuphoria]) into (SvcReq [id=105, name=YUPHORIA])
calling web service:SvcResp [id=104, message=MOTO X Processed successfully]
calling web service:SvcResp [id=105, message=YUPHORIA Processed successfully]
Processed items:2
BATCH JOB FINISHED SUCCESSFULLY
2016-03-22 17:56:01.154  INFO 10932 --- [pool-2-thread-1] o.s.b.c.l.support.SimpleJobLauncher: Job: [FlowJob: [name=processOrderJob]] completed with the following parameters: [{JobID=1458649560996}] and the following status: [COMPLETED]
Job finished with status :COMPLETED

Recommend this on


11 comments:

  1. Nice share.
    Would you please make this project as github repo?
    I think it would be help others.
    Thanks

    ReplyDelete
  2. Thanks for the great share! it's working but I found an issue as those BATCH_* tables did not get updated. So, I just use the @EnableScheduling annotation in my current job configuration and I refer the the auto-initiated launcher instead:

    @Autowired
    private JobLauncher jobLauncher;

    This works with the same results while maintaining the BATCH_* records created.

    ReplyDelete
    Replies
    1. Can you explain me further on this step... even I facing the save issues

      Delete
  3. I have this error when i execute the project

    Caused by: org.springframework.beans.factory.BeanCreationException: Could not autowire field: private org.springframework.batch.core.launch.support.SimpleJobLauncher fr.cc.suivireco.salesmanagement.batch.impl.organismeImport.BatchConfiguratioOrganisme.jobLauncher; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type [org.springframework.batch.core.launch.support.SimpleJobLauncher] found for dependency: expected at least 1 bean which qualifies as autowire candidate for this dependency. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}

    any idea?

    ReplyDelete
    Replies
    1. Please check the package name you added in @ComponentScan({ "com.kswaughs" }) of your main class.
      In my case, all my packages are under "com.kswaughs".

      Delete
  4. Please check the package name you added in @ComponentScan({ "com.kswaughs" }) of your main class.
    In my case, all my packages are under "com.kswaughs".

    ReplyDelete
  5. Thanks for the great walkthrough!

    ReplyDelete
  6. Hi, I am trying to use BatchScheduler in my spring batch project, but getting the below error any help is really appreciated.

    ***************************
    APPLICATION FAILED TO START
    ***************************

    Description:

    Parameter 0 of method mapJobRepositoryFactory in com.barley.batch.BatchScheduler required a bean of type 'org.springframework.batch.support.transaction.ResourcelessTransactionManager' that could not be found.


    Action:

    Consider defining a bean of type 'org.springframework.batch.support.transaction.ResourcelessTransactionManager' in your configuration.

    ReplyDelete
  7. Hi,Thanks for the post . I followed your approach to schedule a spring batch Job but I have multiple jobs which i have to schedule. I am not able to achieve it. Can you please help me with it ?

    ReplyDelete
  8. Reader is in an infinity loop. How to stop reader after the chunk?.

    ReplyDelete
  9. The effectiveness of IEEE Project Domains depends very much on the situation in which they are applied. In order to further improve IEEE Final Year Project Domains practices we need to explicitly describe and utilise our knowledge about software domains of software engineering Final Year Project Domains for CSE technologies. This paper suggests a modelling formalism for supporting systematic reuse of software engineering technologies during planning of software projects and improvement programmes in Project Centers in Chennai for CSE.

    Spring Framework has already made serious inroads as an integrated technology stack for building user-facing applications. Spring Framework Corporate TRaining the authors explore the idea of using Java in Big Data platforms.
    Specifically, Spring Framework provides various tasks are geared around preparing data for further analysis and visualization. Spring Training in Chennai

    ReplyDelete