SpringBatch 配置并行启动Job详解 (八)

文章目录

一、创建并行job

前言：在日常业务中可能需要job并行执行，SpringBatch支持job并行步执行，并且配置简单。

代码已上传GitHub上面地址：https://github.com/FadeHub/spring-boot-learn/tree/master/spring-boot-springbatch

SpringBatch其它文章直通车：

SpringBatch读单个文件(FlatFileItemReader)和写单个文件(FlatFileItemWriter)(一)
SpringBatch顺序读取多文件(MultiResourceItemReader)和顺序写文件(MultiResourceItemWriter)(二)
SpringBatch读数据库(MyBatisPagingItemReader)(三)
SpringBatch读文件(FlatFileItemReader)写据库(MyBatisBatchItemWriter)(四)
SpringBatch 监听器之Job监听器(JobExecutionListener)和Step监听器(StepExecutionListener)(五)
SpringBatch 监听器之Chunk监听器(ChunkListener)和Skip监听器(SkipListener)(六)
SpringBatch 多线程(TaskExecutor)启动Job详解 (七)

一、创建并行job

首先使用FlowBuilder构建一个个小的flow流程，在这个流程里面指定步骤，两个流程flow是并行执行的，下面有两个并行流flow1和flow2，flow1里面有step1 step2先后顺序，flow2有step3，也就是说{step1,step2}一起和step3是并行的：

package com.sl.config;import com.sl.common.CommonFileItemReader;
import com.sl.common.CommonFileItemWriter;
import com.sl.common.CommonMybatisItemReader;
import com.sl.entity.CafeCat;
import com.sl.entity.Cat;
import com.sl.entity.People;
import com.sl.entity.Student;
import com.sl.listener.CatChunkListener;
import com.sl.listener.CatJobListener;
import com.sl.listener.CatStepListener;
import com.sl.processor.CafeCatProcessor;
import com.sl.processor.StudentProcessor;
import org.apache.ibatis.session.SqlSessionFactory;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepScope;
import org.springframework.batch.core.job.builder.FlowBuilder;
import org.springframework.batch.core.job.flow.Flow;
import org.springframework.batch.core.job.flow.support.SimpleFlow;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.task.SimpleAsyncTaskExecutor;
import org.springframework.core.task.TaskExecutor;/*** job并行执行* @author shuliangzhao* @Title: CatFlowConfiguration* @ProjectName spring-boot-learn* @Description: TODO* @date 2019/9/14 20:10*/
@Configuration
@EnableBatchProcessing
public class CatFlowConfiguration {@Autowiredprivate JobBuilderFactory jobBuilderFactory;@Autowiredprivate StepBuilderFactory stepBuilderFactory;@Autowiredprivate CafeCatProcessor cafeCatProcessor;@Autowiredprivate SqlSessionFactory sqlSessionFactory;@Autowiredprivate CatJobListener catJobListener;@Autowiredprivate CatStepListener catStepListener;@Autowiredprivate CatChunkListener catChunkListener;@Autowiredprivate StudentProcessor studentProcessor;@Beanpublic Job catFlowJob() {return jobBuilderFactory.get("catFlowJob").start(splitFlow()).next(catFlowStep()).build().build();}@Beanpublic Step catFlowStep() {return stepBuilderFactory.get("catFlowStep")// .listener(catStepListener).listener(catChunkListener).<Cat, CafeCat>chunk(10).reader(catFlowCommonMybatisItemReader()).processor(cafeCatProcessor).writer(cafeCatFlowCommonFileItemWriter()).taskExecutor(flowTaskExecutor()).throttleLimit(8).build();}@Bean@StepScopepublic CommonMybatisItemReader<Cat> catFlowCommonMybatisItemReader() {return new CommonMybatisItemReader(sqlSessionFactory,Cat.class.getSimpleName());}@Bean@StepScopepublic CommonFileItemWriter<CafeCat> cafeCatFlowCommonFileItemWriter() {return new CommonFileItemWriter<>(CafeCat.class);}@Beanpublic Flow splitFlow() {return new FlowBuilder<SimpleFlow>("splitFlow").split(flowTaskExecutor()).add(flow1(), flow2()).build();}@Beanpublic Flow flow1() {return new FlowBuilder<SimpleFlow>("flow1").start(step1()).next(step2()).build();}@Beanpublic Flow flow2() {return new FlowBuilder<SimpleFlow>("flow2").start(step3()).build();}@Beanpublic Step step3() {return stepBuilderFactory.get("step3").<People, Student>chunk(10).reader(step3flow2CommonFileItemReader()).processor(studentProcessor).writer(step3flow2CommonFileItemWriter()).build();}@Bean@StepScopepublic CommonFileItemReader<People> step3flow2CommonFileItemReader() {return new CommonFileItemReader<>(People.class);}@Bean@StepScopepublic CommonFileItemWriter<Student> step3flow2CommonFileItemWriter() {return new CommonFileItemWriter<>(Student.class);}@Beanpublic Step step1() {return stepBuilderFactory.get("step1").<People, Student>chunk(10).reader(step1flow1CommonFileItemReader()).processor(studentProcessor).writer(step1flow1CommonFileItemWriter()).build();}@Bean@StepScopepublic CommonFileItemReader<People> step1flow1CommonFileItemReader() {return new CommonFileItemReader<>(People.class);}@Bean@StepScopepublic CommonFileItemWriter<Student> step1flow1CommonFileItemWriter() {return new CommonFileItemWriter<>(Student.class);}@Beanpublic Step step2() {return stepBuilderFactory.get("step2").<People, Student>chunk(10).reader(step2flow1CommonFileItemReader()).processor(studentProcessor).writer(step2flow1CommonFileItemWriter()).build();}@Bean@StepScopepublic CommonFileItemReader<People> step2flow1CommonFileItemReader() {return new CommonFileItemReader<>(People.class);}@Bean@StepScopepublic CommonFileItemWriter<Student> step2flow1CommonFileItemWriter() {return new CommonFileItemWriter<>(Student.class);}@Beanpublic TaskExecutor flowTaskExecutor(){return new SimpleAsyncTaskExecutor("spring_batch");}
}

需要指定TaskExecutor 应该使用哪个实现来执行各个流。默认值为 SyncTaskExecutor没有用，需要异步TaskExecutor才能并行运行这些步骤。

SpringBatch 配置并行启动Job详解 (八)相关推荐

SpringBatch 多线程(TaskExecutor)启动Job详解 (七)
文章目录一.SpringBatch配置多线程二.SpringBatch多线程的tasklet配置前言:使用单线程,单个进程作业可以解决许多批处理问题,如果我们想要job执行的速度更快,Sprin ...
JVM 启动参数详解
JVM 启动参数详解 JVM 作为一个通用的虚拟机,我们可以通过启动Java命令时指定不同JVM参数,让 JVM调整自己的运行状态和行为,内存管理和垃圾回收的GC算法等等. 直接通过命令行启动 Jav ...
linux upstart脚本,Linux upstart启动方式详解
Ubuntu从6.10开始逐步用Upstart()代替原来的SysVinit进行服务进程的管理.RHEL(CentOS)也都从版本6开始转用Upstart代替以往的init.d/rcX.d的线性启动方 ...
家用计算机启动过程装载主引导记录,计算机启动过程详解
综述: 计算机启动时经过了哪些过程: 计算机接通电源后,第一步要进行加电自检,也就是POST(Power On Self Test),检查RAM.驱动器等:第二步BIOS会读取活动分区主引导记录的启动 ...
idea 执行java maven,IDEA的run maven方式启动步骤详解
安装jetty插件 1. 找到Plugins,查找jetty插件,安装"IDEA Jetty Runner",安装好后重启IDEA 安装插件:Maven Helper 方法同Jet ...
Linux开启动过程详解
Linux开启动过程详解 Linux启动过程前言: Linux是一种自由和开放源代码的类UNIX操作系统.该操作系统的内核由林纳斯·托瓦兹在1991年10月5日首次发布.在加上用户空间的应用程序之后 ...
centos7 启动流程图_Linux启动过程详解
Linux启动过程详解作者:江远航一.启动流程图如下图1 Linux启动流程图 BIOS ---> MBR ---> Kernel---> Init 二.Linux启动顺序一 ...
Win7系统Visual Studio 2013配置OpenCV3.1图文详解
Win7系统Visual Studio 2013配置OpenCV3.1图文详解 OpenCV3.1对硬件加速和移动开发的支持相对于老版本都有了较大改进,支持新的开发工具,更易于扩展,配置方式也比以前简 ...
U-Boot启动流程详解
参考:U-Boot顶层目录链接脚本文件(u-boot.lds)介绍作者:一只青木呀发布时间: 2020-10-23 13:52:23 网址:https://blog.csdn.net/weixin ...

SpringBatch 配置并行启动Job详解 (八)

文章目录

一、创建并行job

SpringBatch 配置并行启动Job详解 (八)相关推荐

最新文章

热门文章