SpringBatch 实列学习《一》

通过Spring batch的官方文档，学习springbatch, 记录每个实列。

本案例是官方提供的第一个入门实列

官方文档地址

https://spring.io/guides/gs/batch-processing/#scratch

源码地址

https://gitee.com/bseaworkspace/study_java_web/tree/master/springbatchdemobasic

代码结构

业务数据

Typically, your customer or a business analyst supplies a spreadsheet. For this simple example, you can find some made-up data in src/main/resources/sample-data.csv:

比如用户提供了一个需要处理的表格数据。

Jill,Doe
Joe,Doe
Justin,Doe
Jane,Doe
John,Doe

这个表格数据每一行包含了名和姓并且用逗号隔开。

接下来，我们需要准备一个sql 用来创建需要的数据表 src/main/resources/schema-all.sql:

DROP TABLE people IF EXISTS;CREATE TABLE people  (person_id BIGINT IDENTITY NOT NULL PRIMARY KEY,first_name VARCHAR(20),last_name VARCHAR(20)
);

Spring Boot runs schema-@@platform@@.sql automatically during startup. -all is the default for all platforms.

Spring Boot提供两种方法来定义数据库的表结构以及添加数据。

使用Hibernate提供的工具来创建表结构，该机制会自动搜索@Entity实体对象并创建对应的表，然后使用import.sql文件导入测试数据；

利用旧的Spring JDBC，通过schema.sql文件定义数据库的表结构、通过data.sql导入测试数据。

这里采用了Spring JDBC的方式，在springboot项目启动的时候，自动创建相关的表。

Spring Boot可以自动创建DataSource的模式（DDL脚本）并初始化它（DML脚本），并从标准的位置 schema.sql 和 data.sql （位于classpath根目录）加载SQL，脚本的位置可以通过设置 spring.datasource.schema 和 spring.datasource.data 来改变。此外，Spring Boot将加载 schema-${platform}.sql和 data-${platform}.sql 文件（如果存在），在这里 platform 是 spring.datasource.platform 的值，比如，可以将它设置为数据库的供应商名称（ hsqldb , h2 , oracle , mysql , postgresql 等）。
Spring Boot默认启用Spring JDBC初始化快速失败特性，所以如果脚本导致异常产生，那应用程序将启动失败。能通过设置spring.datasource.continue-on-error的值来控制是否继续。一旦应用程序成熟并被部署了很多次，那该设置就很有用，例如，插入失败时意味着数据已经存在，也就没必要阻止应用继续运行。
如果想要在一个JPA应用中使用 schema.sql ，那如果Hibernate试图创建相同的表， ddl-auto=create-drop 将导致错误产生。为了避免那些错误，可以将 ddl-auto 设置为""或 none 。
最后要提的一点是，spring.datasource.initialize=false 可以阻止数据初始化。

开始代码实现

设置Maven的pom文件

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"><modelVersion>4.0.0</modelVersion><parent><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-parent</artifactId><version>2.4.3</version><relativePath/> <!-- lookup parent from repository --></parent><groupId>com.example</groupId><artifactId>batch-processing</artifactId><version>0.0.1-SNAPSHOT</version><name>batch-processing</name><description>Demo project for Spring Boot</description><properties><java.version>1.8</java.version></properties><dependencies><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-batch</artifactId></dependency><dependency><groupId>org.hsqldb</groupId><artifactId>hsqldb</artifactId><scope>runtime</scope></dependency><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-test</artifactId><scope>test</scope></dependency><dependency><groupId>org.springframework.batch</groupId><artifactId>spring-batch-test</artifactId><scope>test</scope></dependency></dependencies><build><plugins><plugin><groupId>org.springframework.boot</groupId><artifactId>spring-boot-maven-plugin</artifactId></plugin></plugins></build></project>

业务实体类

Now that you can see the format of data inputs and outputs, you can write code to represent a row of data, as the following example shows:

用来定义数据的输入和输出的格式，一个实体类的对象，就对于前面表格数据的一行数据。

package com.xsz.entity;
public class Person {private String lastName;private String firstName;public Person() {}public Person(String firstName, String lastName) {this.firstName = firstName;this.lastName = lastName;}public void setFirstName(String firstName) {this.firstName = firstName;}public String getFirstName() {return firstName;}public String getLastName() {return lastName;}public void setLastName(String lastName) {this.lastName = lastName;}@Overridepublic String toString() {return "firstName: " + firstName + ", lastName: " + lastName;}}

Create an Intermediate Processor 创建中间处理类

A common paradigm in batch processing is to ingest data, transform it, and then pipe it out somewhere else. Here, you need to write a simple transformer that converts the names to uppercase. The following listing

这个中间处理类，主要是把输入对象的姓名属性的值，转换成大写。

package com.xsz.processor;import com.xsz.entity.Person;import org.slf4j.Logger;import org.slf4j.LoggerFactory;import org.springframework.batch.item.ItemProcessor;public class PersonItemProcessor implements ItemProcessor<Person, Person> {private static final Logger log = LoggerFactory.getLogger(PersonItemProcessor.class);@Overridepublic Person process(final Person person) throws Exception {final String firstName = person.getFirstName().toUpperCase();final String lastName = person.getLastName().toUpperCase();final Person transformedPerson = new Person(firstName, lastName);log.info("Converting (" + person + ") into (" + transformedPerson + ")");return transformedPerson;}}

Put Together a Batch Job 创建批处理job

 @Configuration  放在java类上面，表示这个是配置类，代替xml文件

```
@EnableBatchProcessing
```

@EnableBatchProcessing
这个注解的作用，和spring 家庭中的@Enable* 系列注解功能很类似。顾名思义，就是让我们可以运行Spring Batch。

在配置类上打上这个注解，spring 会自动帮我们生成一系列与spring batch 运行有关的bean，并交给spring容器管理，而当我们需要这些beans时，只需要用一个@autowired就可以实现注入了。

自动生成的bean及名称如下：

JobRepository - bean name "jobRepository"

JobLauncher - bean name "jobLauncher"

JobRegistry - bean name "jobRegistry"

PlatformTransactionManager - bean name "transactionManager"

JobBuilderFactory - bean name "jobBuilders"

StepBuilderFactory - bean name "stepBuilders"

@EnableBatchProcessing 背后所调用的接口是BatchConfigurer

我们可以改变@EnableBatchProcessing 给我们的对象。

比如，我们想让返回的jobRepository用上我们自定义的数据源，想让元数据的表前缀变成SPRING_BATCH_，而不是默认的BATCH_前缀，@EnableBatchProcessing 提供了让我们覆写的接口。

The first chunk of code defines the input, processor, and output. 一个chunk 需要配置对应的输入，中间处理，输出

reader() creates an ItemReader. It looks for a file called sample-data.csv and parses each line item with enough information to turn it into a Person. reader()方法里面定义了数据输入的规则，读取csv的文件内容，每行对应一个person对象
processor() creates an instance of the PersonItemProcessor that you defined earlier, meant to convert the data to upper case. 把reader读到jvm里面的person对象属性值变成大写。
writer(DataSource) creates an ItemWriter. This one is aimed at a JDBC destination and automatically gets a copy of the dataSource created by @EnableBatchProcessing. It includes the SQL statement needed to insert a single Person, driven by Java bean properties.

package com.xsz.config;import com.xsz.entity.Person;
import com.xsz.processor.PersonItemProcessor;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.support.RunIdIncrementer;
import org.springframework.batch.item.database.BeanPropertyItemSqlParameterSourceProvider;
import org.springframework.batch.item.database.JdbcBatchItemWriter;
import org.springframework.batch.item.database.builder.JdbcBatchItemWriterBuilder;
import org.springframework.batch.item.file.FlatFileItemReader;
import org.springframework.batch.item.file.builder.FlatFileItemReaderBuilder;
import org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.ClassPathResource;import javax.sql.DataSource;@Configuration
@EnableBatchProcessing
public class BatchConfiguration {@Autowiredpublic JobBuilderFactory jobBuilderFactory;@Autowiredpublic StepBuilderFactory stepBuilderFactory;//    ...
@Bean
public FlatFileItemReader<Person> reader() {return new FlatFileItemReaderBuilder<Person>().name("personItemReader").resource(new ClassPathResource("sample-data.csv")).delimited().names(new String[]{"firstName", "lastName"}).fieldSetMapper(new BeanWrapperFieldSetMapper<Person>() {{setTargetType(Person.class);}}).build();
}@Beanpublic PersonItemProcessor processor() {return new PersonItemProcessor();}@Beanpublic JdbcBatchItemWriter<Person> writer(DataSource dataSource) {return new JdbcBatchItemWriterBuilder<Person>().itemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<>()).sql("INSERT INTO people (first_name, last_name) VALUES (:firstName, :lastName)").dataSource(dataSource).build();}@Beanpublic Job importUserJob(JobCompletionNotificationListener listener, Step step1) {return jobBuilderFactory.get("importUserJob").incrementer(new RunIdIncrementer()).listener(listener).flow(step1).end().build();}@Beanpublic Step step1(JdbcBatchItemWriter<Person> writer) {return stepBuilderFactory.get("step1").<Person, Person> chunk(10).reader(reader()).processor(processor()).writer(writer).build();}
}