分库分表之_分库分表 + 复杂查询
前言
Github:https://github.com/HealerJean
博客:http://blog.healerjean.com
代码配置暂时和和分库分表之_分库分表相同。但是为了测试下面的join查询将user表的数量调整到了和company表的数量一致,以及给company添加了一个新的字段ref_user_id
1、开始Demo
1.1、hlj-08-sharding_db_table-range_group.sql
drop database if exists ds_0;
create database ds_0 character set 'utf8' collate 'utf8_general_ci';
use ds_0;drop table if exists user_0;
create table `user_0`
(id bigint(20) unsigned not null,city varchar(20) not null default '',name varchar(20) not null default '',age int(11) not null default 0,status int(10) not null default '0' comment '状态',create_time datetime not null default current_timestamp comment '创建时间',update_time datetime not null default current_timestamp on update current_timestamp comment '修改时间',primary key (id)
) engine = innodbdefault charset = utf8;drop table if exists user_1;
create table `user_1`
(id bigint(20) unsigned not null,city varchar(20) not null default '',name varchar(20) not null default '',age int(11) not null default 0,status int(10) not null default '0' comment '状态',create_time datetime not null default current_timestamp comment '创建时间',update_time datetime not null default current_timestamp on update current_timestamp comment '修改时间',primary key (id)
) engine = innodbdefault charset = utf8;drop table if exists company_0;
create table `company_0`
(id bigint(20) unsigned not null comment '主键',ref_user_id bigint(20) unsigned not null comment '主键',name varchar(20) not null default '' comment '企业名称',company_name_english varchar(128) not null default '' comment '企业英文名称',status int(10) not null default '0' comment '状态',create_time datetime not null default current_timestamp comment '创建时间',update_time datetime not null default current_timestamp on update current_timestamp comment '修改时间',primary key (id)
) engine = innodbdefault charset = utf8;drop table if exists company_1;
create table `company_1`
(id bigint(20) unsigned not null comment '主键',ref_user_id bigint(20) unsigned not null comment '主键',name varchar(20) not null default '' comment '企业名称',company_name_english varchar(128) not null default '' comment '企业英文名称',status int(10) not null default '0' comment '状态',create_time datetime not null default current_timestamp comment '创建时间',update_time datetime not null default current_timestamp on update current_timestamp comment '修改时间',primary key (id)
) engine = innodbdefault charset = utf8;CREATE TABLE `demo_entity`
(id bigint(20) unsigned NOT NULL COMMENT '主键',`name` varchar(64) NOT NULL,`phone` varchar(20) DEFAULT '' COMMENT '手机号',`email` varchar(64) DEFAULT '' COMMENT '邮箱',`age` int(10) DEFAULT NULL,`status` varchar(8) NOT NULL COMMENT '状态',`create_user` bigint(16) unsigned DEFAULT NULL COMMENT '创建人',`create_name` varchar(64) DEFAULT '' COMMENT '创建人名称',`create_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',`update_user` bigint(16) unsigned DEFAULT NULL COMMENT '更新人',`update_name` varchar(64) DEFAULT '' COMMENT '更新人名称',`update_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '更新时间',PRIMARY KEY (id)
) ENGINE = InnoDBDEFAULT CHARSET = utf8;drop database if exists ds_1;
create database ds_1 character set 'utf8' collate 'utf8_general_ci';
use ds_1;drop table if exists user_0;
create table `user_0`
(id bigint(20) unsigned not null,city varchar(20) not null default '',name varchar(20) not null default '',age int(11) not null default 0,status int(10) not null default '0' comment '状态',create_time datetime not null default current_timestamp comment '创建时间',update_time datetime not null default current_timestamp on update current_timestamp comment '修改时间',primary key (id)
) engine = innodbdefault charset = utf8;drop table if exists user_1;
create table `user_1`
(id bigint(20) unsigned not null,city varchar(20) not null default '',name varchar(20) not null default '',age int(11) not null default 0,status int(10) not null default '0' comment '状态',create_time datetime not null default current_timestamp comment '创建时间',update_time datetime not null default current_timestamp on update current_timestamp comment '修改时间',primary key (id)
) engine = innodbdefault charset = utf8;drop table if exists company_0;
create table `company_0`
(id bigint(20) unsigned not null comment '主键',ref_user_id bigint(20) unsigned not null comment '主键',name varchar(20) not null default '' comment '企业名称',company_name_english varchar(128) not null default '' comment '企业英文名称',status int(10) not null default '0' comment '状态',create_time datetime not null default current_timestamp comment '创建时间',update_time datetime not null default current_timestamp on update current_timestamp comment '修改时间',primary key (id)
) engine = innodbdefault charset = utf8;drop table if exists company_1;
create table `company_1`
(id bigint(20) unsigned not null comment '主键',ref_user_id bigint(20) unsigned not null comment '主键',name varchar(20) not null default '' comment '企业名称',company_name_english varchar(128) not null default '' comment '企业英文名称',status int(10) not null default '0' comment '状态',create_time datetime not null default current_timestamp comment '创建时间',update_time datetime not null default current_timestamp on update current_timestamp comment '修改时间',primary key (id)
) engine = innodbdefault charset = utf8;
1.1.1、数据库图文
1.2、依赖
<!--shardingsphere-->
<dependency><groupId>org.apache.shardingsphere</groupId><artifactId>sharding-jdbc-spring-boot-starter</artifactId><version>4.0.0-RC1</version>
</dependency>
1.3、配置文件:application.properties
server.port=8888# 配置 mybatis的一些配置,也可以在 application.properties 中配置,如果配置了就不需要了mybatis.xml
#mybatis-plus.config-location=classpath:mybatis.xml
#Maven 多模块项目的扫描路径需以 classpath*: 开头 (即加载多个 jar 包下的 XML 文件)
mybatis-plus.mapper-locations=classpath*:mapper/*.xml
mybatis-plus.type-aliases-package=com.healerjean.proj.pojo
##主键类型 0:"数据库ID自增,非常大", 1:"用户输入ID(如果用户不输入,则默认是0)",2:"全局唯一ID (数字类型唯一ID)", 3:"全局唯一ID UUID";
mybatis-plus.id-type: 0
#字段策略 0:"忽略判断",1:"非 NULL 判断"),2:"非空判断"
mybatis-plus.field-strategy: 2
#数据库大写下划线转换
mybatis-plus.capital-mode: true
mybatis-plus.refresh-mapper: true# #当遇到同样名字的时候,是否允许覆盖注册
spring.main.allow-bean-definition-overriding=true
# 显示SQL
spring.shardingsphere.props.sql.show=true##############################
## 分库分表
#############################
spring.shardingsphere.datasource.names=ds0,ds1# 数据源
spring.shardingsphere.datasource.ds0.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.ds0.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds0.url=jdbc:mysql://localhost:3306/ds_0?serverTimezone=CTT&useUnicode=true&characterEncoding=utf-8&allowMultiQueries=true
spring.shardingsphere.datasource.ds0.username=root
spring.shardingsphere.datasource.ds0.password=123456spring.shardingsphere.datasource.ds1.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.ds1.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds1.url=jdbc:mysql://localhost:3306/ds_1?serverTimezone=CTT&useUnicode=true&characterEncoding=utf-8&allowMultiQueries=true
spring.shardingsphere.datasource.ds1.username=root
spring.shardingsphere.datasource.ds1.password=123456# 分库配置
#spring.shardingsphere.sharding.default-database-strategy.inline.sharding-column=id
#spring.shardingsphere.sharding.default-database-strategy.inline.algorithm-expression=ds$->{id % 2}
spring.shardingsphere.sharding.default-database-strategy.standard.sharding-column=id
spring.shardingsphere.sharding.default-database-strategy.standard.precise-algorithm-class-name=com.healerjean.proj.config.datasource.CustomShardingDBAlgorithm# user company 分表
# user_0,user_1,user_2(自定义分表算法)
spring.shardingsphere.sharding.tables.user.actual-data-nodes=ds$->{0..1}.user_$->{0..1}
spring.shardingsphere.sharding.tables.user.table-strategy.standard.sharding-column=id
spring.shardingsphere.sharding.tables.user.table-strategy.standard.precise-algorithm-class-name=com.healerjean.proj.config.datasource.CustomShardingTableAlgorithm
# company_0,company_1 (inline分表策略 表达式 id%2)
spring.shardingsphere.sharding.tables.company.actual-data-nodes=ds$->{0..1}.company_$->{0..1}
spring.shardingsphere.sharding.tables.company.database-strategy.inline.sharding-column=ref_user_id
spring.shardingsphere.sharding.tables.company.database-strategy.inline.algorithm-expression=ds${ref_user_id.longValue() % 2}
spring.shardingsphere.sharding.tables.company.table-strategy.inline.sharding-column=ref_user_id
spring.shardingsphere.sharding.tables.company.table-strategy.inline.algorithm-expression=company_${ref_user_id.longValue() % 2}## 默认数据源指定(不分库的表)
spring.shardingsphere.sharding.default-data-source-name=ds0
1.4、具体测试方法和类
1.4.1、实体类
1.4.1.1、User.java
@Data
@Accessors(chain = true)
public class User implements Serializable {private static final long serialVersionUID = 1L;/** 主键 */private Long id;private String name;private String city;private Integer age;private String status;private Date createTime;private Date updateTime;
}
1.4.1.2、Company.java
@Data
public class Company {private Long id;private Long refUserId;private String name;private String companyNameEnglish;private String status;private Date createTime;private Date updateTime;
}
1.4.1.3、DemoEntity.java
@Data
@Accessors(chain = true)
public class DemoEntity implements Serializable {private static final long serialVersionUID = 1L;private Long id;private String name;private String phone;private String email;private Integer age;private String status;private Long createUser;private String createName;private java.util.Date createTime;private Long updateUser;private String updateName;private java.util.Date updateTime;}
1.4.1.4、新对象 UserRefCompany
@Data
public class UserRefCompany {/** 主键 */private Long userId;private String name;private String city;private String status;private Long companyId;private String companyName;private String companyNameEnglish;private Integer avgAge;private Integer sumAge;}
1.4.2、DTO数据
1.4.2.1、UserDTO.java
@Data
@Accessors(chain = true)
@ApiModel(value = "demo实体类")
@JsonInclude(JsonInclude.Include.NON_NULL)
public class UserDTO {@ApiModelProperty(value = "主键", hidden = true)@JsonSerialize(using = JsonLongSerializer.class )private Long id;@ApiModelProperty(value = "姓名")@NotBlank(message = "姓名不能为空", groups = ValidateGroup.HealerJean.class)private String name;@ApiModelProperty(value = "城市")private String city;@ApiModelProperty(value = "年龄")private Integer age;@ApiModelProperty(value = "状态", hidden = true)private String status;@ApiModelProperty(value = "创建时间", hidden = true)@JsonFormat(pattern = DateUtils.YYYY_MM_dd_HH_mm_ss, timezone = "GMT+8")private Date createTime;@ApiModelProperty(value = "修改时间", hidden = true)@JsonFormat(pattern = DateUtils.YYYY_MM_dd_HH_mm_ss, timezone = "GMT+8")private Date updateTime;private Integer pageNow ;private Integer pageSize ;}
1.4.2.2、CompanyDTO.java
@Data
public class CompanyDTO {@JsonSerialize(using = JsonLongSerializer.class)private Long id;private Long refUserId;private String name;private String companyNameEnglish;private String status;@ApiModelProperty(value = "创建时间", hidden = true)@JsonFormat(pattern = DateUtils.YYYY_MM_dd_HH_mm_ss, timezone = "GMT+8")private Date createTime;@ApiModelProperty(value = "修改时间", hidden = true)@JsonFormat(pattern = DateUtils.YYYY_MM_dd_HH_mm_ss, timezone = "GMT+8")private Date updateTime;
}
1.4.2.3、DemoDTO.java
@Data
@Accessors(chain = true)
@ApiModel(value = "demo实体类")
@JsonInclude(JsonInclude.Include.NON_NULL)
public class DemoDTO extends PageQuery {@JsonSerialize(using = JsonLongSerializer.class )private Long id;@ApiModelProperty(value = "姓名")@NotBlank(message = "姓名不能为空", groups = ValidateGroup.HealerJean.class)private String name;@ApiModelProperty(value = "年龄")private Integer age;@ApiModelProperty(value = "手机号")private String phone;@ApiModelProperty(value = "邮箱")private String email;@ApiModelProperty(value = "是否删除,10可用,99删除 ", hidden = true)private String status;@ApiModelProperty(value = "创建人", hidden = true)private Long createUser;@ApiModelProperty(value = "创建人名字", hidden = true)private String createName;@ApiModelProperty(value = "创建时间", hidden = true)private java.util.Date createTime;@ApiModelProperty(value = "更新人", hidden = true)private Long updateUser;@ApiModelProperty(value = "更新人名称", hidden = true)private String updateName;@ApiModelProperty(hidden = true)private java.util.Date updateTime;}
1.4.3、Mapper
1.4.3.1、UserMapper.java
public interface UserMapper extends BaseMapper<User> {}
1.4.3.1、CompanyMapper.java
public interface CompanyMapper extends BaseMapper<Company> {}
1.4.3.1、DemoEntityMapper.java
public interface DemoEntityMapper extends BaseMapper<DemoEntity> {}
1.4.4、Service
1.4.4.1、 UserService.java
public interface UserService {UserDTO insert(UserDTO userDTO);UserDTO findById(Long id);List<UserDTO> list();}
1.4.4.2、 CompanyService.java
public interface CompanyService {CompanyDTO insert(CompanyDTO companyDTO);CompanyDTO findById(Long id);List<CompanyDTO> list();
}
1.4.4.3、 DemoEntityService.java
public interface DemoEntityService {DemoDTO insert(DemoDTO demoEntity);DemoDTO findById(Long id);List<DemoDTO> list();}
1.4.5、ServiceImpl.java
1.4.5.1、UserServiceImpl.java
@Service
@Slf4j
public class UserServiceImpl implements UserService {@Resourceprivate UserMapper userMapper;@Overridepublic UserDTO insert(UserDTO userDTO) {User user = BeanUtils.dtoToUserDTO(userDTO);user.setStatus(StatusEnum.生效.code);userMapper.insert(user);userDTO.setId(user.getId());return userDTO;}@Overridepublic UserDTO findById(Long id) {User user = userMapper.selectById(id);return user == null ? null : BeanUtils.userToDTO(user);}@Overridepublic List<UserDTO> list() {List<User> users = userMapper.selectList(null);List<UserDTO> list = null;if (!EmptyUtil.isEmpty(users)) {list = users.stream().map(BeanUtils::userToDTO).collect(Collectors.toList());}return list;}}
1.4.5.2、CompanyServiceImpl.java
@Service
public class CompanyServiceImpl implements CompanyService {@Resourceprivate CompanyMapper companyMapper;@Overridepublic CompanyDTO insert(CompanyDTO companyDTO) {Company company = BeanUtils.dtoToCompany(companyDTO);company.setStatus(StatusEnum.生效.code);companyMapper.insert(company);companyDTO.setId(company.getId());return companyDTO;}@Overridepublic CompanyDTO findById(Long id) {Company company = companyMapper.selectById(id);return company == null ? null : BeanUtils.companyToDTO(company);}@Overridepublic List<CompanyDTO> list() {List<Company> companys = companyMapper.selectList(null);List<CompanyDTO> list = null;if (!EmptyUtil.isEmpty(companys)) {list = companys.stream().map(BeanUtils::companyToDTO).collect(Collectors.toList());}return list;}
}
1.4.5.3、DemoEntityServiceImpl.java
@Service
@Slf4j
public class DemoEntityServiceImpl implements DemoEntityService {@Resourceprivate DemoEntityMapper demoEntityMapper;@Resourceprivate CompanyService companyService;@Resourceprivate UserService userService;@Overridepublic DemoDTO insert(DemoDTO demoDTO) {DemoEntity demoEntity = BeanUtils.dtoToDemo(demoDTO);demoEntity.setStatus(StatusEnum.生效.code);demoEntityMapper.insert(demoEntity);demoDTO.setId(demoEntity.getId());return demoDTO;}@Overridepublic DemoDTO findById(Long id) {DemoEntity demoEntity = demoEntityMapper.selectById(id);return demoEntity == null ? null : BeanUtils.demoToDTO(demoEntity);}@Overridepublic List<DemoDTO> list() {List<DemoDTO> collect = null;List<DemoEntity> list = demoEntityMapper.selectList(null);if (!EmptyUtil.isEmpty(list)) {collect = list.stream().map(BeanUtils::demoToDTO).collect(Collectors.toList());}return collect;}}
1.4.6、Controller
1.4.6.1、UserController.java
@ApiResponses(value = {@ApiResponse(code = 200, message = "访问正常"),@ApiResponse(code = 301, message = "逻辑错误"),@ApiResponse(code = 500, message = "系统错误"),@ApiResponse(code = 401, message = "未认证"),@ApiResponse(code = 403, message = "禁止访问"),@ApiResponse(code = 404, message = "url错误")
})
@Api(description = "demo控制器")
@Controller
@RequestMapping("hlj/demo")
@Slf4j
public class UserController {@Autowiredprivate UserService userService;@ApiOperation(value = "insert",notes = "insert",consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,produces = MediaType.APPLICATION_JSON_VALUE,response = UserDTO.class)@PostMapping(value = "insert", produces = "application/json; charset=utf-8")@ResponseBodypublic ResponseBean insert(UserDTO userDTO) {log.info("样例--------mybaits-plus添加demo实体------数据信息{}", userDTO);String validate = ValidateUtils.validate(userDTO, ValidateGroup.HealerJean.class);if (!validate.equals(CommonConstants.COMMON_SUCCESS)) {throw new BusinessException(ResponseEnum.参数错误, validate);}return ResponseBean.buildSuccess(userService.insert(userDTO));}@ApiOperation(notes = "findById",value = "findById",consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,produces = MediaType.APPLICATION_JSON_VALUE,response = UserDTO.class)@ApiImplicitParams({@ApiImplicitParam(name = "id", value = "demo主键", required = true, paramType = "path", dataType = "long"),})@GetMapping("findById/{id}")@ResponseBodypublic ResponseBean findById(@PathVariable Long id) {log.info("样例--------findById------数据:id:{}", id);return ResponseBean.buildSuccess(userService.findById(id));}@ApiOperation(notes = "list",value = "list",consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,produces = MediaType.APPLICATION_JSON_VALUE,response = UserDTO.class)@GetMapping("list")@ResponseBodypublic ResponseBean list() {log.info("样例--------list------");return ResponseBean.buildSuccess(userService.list());}}
1.4.6.2、CompanyController.java
@ApiResponses(value = {@ApiResponse(code = 200, message = "访问正常"),@ApiResponse(code = 301, message = "逻辑错误"),@ApiResponse(code = 500, message = "系统错误"),@ApiResponse(code = 401, message = "未认证"),@ApiResponse(code = 403, message = "禁止访问"),@ApiResponse(code = 404, message = "url错误")
})
@Api(description = "demo控制器")
@Controller
@RequestMapping("hlj/company")
@Slf4j
public class CompanyController {@Autowiredprivate CompanyService companyService;@ApiOperation(value = "insert",notes = "insert",consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,produces = MediaType.APPLICATION_JSON_VALUE,response = UserDTO.class)@PostMapping(value = "insert", produces = "application/json; charset=utf-8")@ResponseBodypublic ResponseBean insert(CompanyDTO companyDTO) {log.info("user--------insert------请求参数:{}", companyDTO);return ResponseBean.buildSuccess(companyService.insert(companyDTO));}@ApiOperation(notes = "findById",value = "findById",consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,produces = MediaType.APPLICATION_JSON_VALUE,response = UserDTO.class)@ApiImplicitParams({@ApiImplicitParam(name = "id", value = "demo主键", required = true, paramType = "path", dataType = "long"),})@GetMapping("findById/{id}")@ResponseBodypublic ResponseBean findById(@PathVariable Long id) {log.info("company--------findById------id:{}", id);return ResponseBean.buildSuccess(companyService.findById(id));}@ApiOperation(notes = "list",value = "list",consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,produces = MediaType.APPLICATION_JSON_VALUE,response = UserDTO.class)@GetMapping("list")@ResponseBodypublic ResponseBean list() {log.info("company--------list------");return ResponseBean.buildSuccess(companyService.list());}}
1.4.6.3、DemoController.java
@ApiResponses(value = {@ApiResponse(code = 200, message = "访问正常"),@ApiResponse(code = 301, message = "逻辑错误"),@ApiResponse(code = 500, message = "系统错误"),@ApiResponse(code = 401, message = "未认证"),@ApiResponse(code = 403, message = "禁止访问"),@ApiResponse(code = 404, message = "url错误")
})
@Api(description = "demo控制器")
@Controller
@RequestMapping("hlj/demo")
@Slf4j
public class DemoController {@Autowiredprivate DemoEntityService demoEntityService;@ApiOperation(value = "insert",notes = "insert",consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,produces = MediaType.APPLICATION_JSON_VALUE,response = UserDTO.class)@PostMapping(value = "insert", produces = "application/json; charset=utf-8")@ResponseBodypublic ResponseBean insert(DemoDTO demoDTO) {log.info("demo--------insert------请求参数:{}", demoDTO);return ResponseBean.buildSuccess(demoEntityService.insert(demoDTO));}@ApiOperation(notes = "findById",value = "findById",consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,produces = MediaType.APPLICATION_JSON_VALUE,response = UserDTO.class)@ApiImplicitParams({@ApiImplicitParam(name = "id", value = "demo主键", required = true, paramType = "path", dataType = "long"),})@GetMapping("findById/{id}")@ResponseBodypublic ResponseBean findById(@PathVariable Long id) {log.info("demo--------findById------id:{}", id);return ResponseBean.buildSuccess(demoEntityService.findById(id));}@ApiOperation(notes = "list",value = "list",consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,produces = MediaType.APPLICATION_JSON_VALUE,response = UserDTO.class)@GetMapping("list")@ResponseBodypublic ResponseBean list() {log.info("demo--------list------");return ResponseBean.buildSuccess(demoEntityService.list());}}
1.4.6、自定义分表算法 CustomShardingTableAlgorithm
@Slf4j
public class CustomShardingTableAlgorithm implements PreciseShardingAlgorithm<Long> {@Overridepublic String doSharding(Collection<String> availableTargetNames, PreciseShardingValue<Long> shardingValue) {for (String tableName : availableTargetNames) {if (tableName.endsWith(shardingValue.getValue() % 3 + "")) {log.info("表为:{}, 主键为:{}, 最终被分到的表为:{}", availableTargetNames, shardingValue, tableName);return tableName;}}throw new IllegalArgumentException();}
}
1.4.7、自定义分库算法:CustomShardingDBAlgorithm
@Slf4j
public class CustomShardingDBAlgorithm implements PreciseShardingAlgorithm<Long> {@Overridepublic String doSharding(Collection<String> availableTargetNames, PreciseShardingValue<Long> shardingValue) {for (String dbName : availableTargetNames) {if (dbName.endsWith(shardingValue.getValue() % 2 + "")) {log.info("库为:{}, 主键为:{}, 最终被分到的库为:【{}】", availableTargetNames, shardingValue, dbName);return dbName;}}throw new IllegalArgumentException();}
}
2、开始测试
基本的增删改成就不测试了,我们这里主要是测试一些分库后可能出问题的一些语句
2.1、limit查询:成功
2.1.1、代码如下
2.1.1.1、UserService.java
public interface UserService {List<UserDTO> limit(UserDTO userDTO);}
2.1.1.2、UserServiceImpl.java
@Service
@Slf4j
public class UserServiceImpl implements UserService {/*** limit 成功*/@Overridepublic List<UserDTO> limit(UserDTO userDTO) {Wrapper<User> userWrapper = new QueryWrapper<User>().lambda().orderByDesc(User::getCreateTime).last("limit " + userDTO.getPageNow() + ", " + userDTO.getPageSize());List<User> users = userMapper.selectList(userWrapper);List<UserDTO> list = null;if (!EmptyUtil.isEmpty(users)) {list = users.stream().map(BeanUtils::userToDTO).collect(Collectors.toList());}return list;}}
2.1.1.3、UserController.java
@Controller
@RequestMapping("hlj/user")
@Slf4j
public class UserController {@ApiOperation(notes = "limit",value = "limit",consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,produces = MediaType.APPLICATION_JSON_VALUE,response = UserDTO.class)@GetMapping("limit")@ResponseBodypublic ResponseBean limit(UserDTO userDTO) {log.info("demo--------limit------");return ResponseBean.buildSuccess(userService.limit(userDTO));}}
2.1.2、访问测试
现在所有的User表数据汇总:
name | city | age | createTime |
---|---|---|---|
a | 北京 | 1 | 2020-03-30 18:15:14 |
b | 北京 | 2 | 2020-03-30 18:16:14 |
c | 山西 | 3 | 2020-03-30 18:16:20 |
d | 山西 | 4 | 2020-03-30 18:16:27 |
e | 河北 | 5 | 2020-03-30 18:16:38 |
f | 江苏 | 6 | 2020-03-30 18:16:46 |
2.1.2.1、访问:pageNow:0,pageSize:2
{"success": true,"result": [{"id": "1244569246752800769","name": "f","city": "江苏","age": 6,"status": "10","createTime": "2020-03-30 18:16:46","updateTime": "2020-03-30 18:16:46"},{"id": "1244569214813175810","name": "e","city": "河北","age": 5,"status": "10","createTime": "2020-03-30 18:16:38","updateTime": "2020-03-30 18:16:38"}],"msg": "","code": 200,"date": "1585563469572"
}
控制台sql日志
# 日志汇总
Logic SQL:
SELECT id,name,city,age,status,create_time,update_time FROM user ORDER BY create_time DESC limit 0, 2 # 数据库日志
Actual SQL: ds0 :::
SELECT id,name,city,age,status,create_time,update_time FROM user_0
ORDER BY create_time DESC limit 0, 2 ShardingSphere-SQL.log[89]Actual SQL: ds0 :::
SELECT id,name,city,age,status,create_time,update_time FROM user_1 ORDER BY create_time DESC limit 0, 2Actual SQL: ds1 :::
SELECT id,name,city,age,status,create_time,update_time FROM user_0 ORDER BY create_time DESC limit 0, 2 Actual SQL: ds1 :::
SELECT id,name,city,age,status,create_time,update_time FROM user_1
ORDER BY create_time DESC limit 0, 2
2.1.2.2、访问:pageNow:3,pageSize:1
{"success": true,"result": [{"id": "1244569141396078594","name": "c","city": "山西","age": 3,"status": "10","createTime": "2020-03-30 18:16:20","updateTime": "2020-03-30 18:16:20"}],"msg": "","code": 200,"date": "1585563506891"
}
控制台sql日志
# 日志汇总
Actual SQL:
SELECT id,name,city,age,status,create_time,update_time FROM user ORDER BY create_time DESC limit 3, 1 # 数据库日志
Actual SQL: ds0 :::
SELECT id,name,city,age,status,create_time,update_time FROM user_0 ORDER BY create_time DESC limit 0, 4Actual SQL: ds0 :::
SELECT id,name,city,age,status,create_time,update_time FROM user_1 ORDER BY create_time DESC limit 0, 4Actual SQL: ds1 :::
SELECT id,name,city,age,status,create_time,update_time FROM user_0 ORDER BY create_time DESC limit 0, 4Actual SQL: ds1 :::
SELECT id,name,city,age,status,create_time,update_time FROM user_1 ORDER BY create_time DESC limit 0, 4
2.1.3、归纳总结:
成功,可以看到分表向每个数据库的每个表中发送了sql语句
2.2、groupBy + 函数 :成功
2.2.1、代码如下
2.2.1.1、UserMapper.java
public interface UserMapper extends BaseMapper<User> {List<UserRefCompany> groupByCity();
}
2.2.1.2、UserMapper.xml
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd" >
<mapper namespace="com.healerjean.proj.dao.mapper.UserMapper"><select id="groupByCity" resultType="com.healerjean.proj.pojo.UserRefCompany">select city , sum(age) as sumAge, avg(age) as avgAge from user group by city</select></mapper>
2.2.1.3、UserService.java
@Service
@Slf4j
public class UserServiceImpl implements UserService {/*** groupBy 成功*/@Overridepublic List<UserRefCompany> group() {List<UserRefCompany> list = userMapper.groupByCity();return list;}}
2.2.1.4、UserController.java
@Controller
@RequestMapping("hlj/user")
@Slf4j
public class UserController {@ApiOperation(notes = "group",value = "group",consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,produces = MediaType.APPLICATION_JSON_VALUE,response = UserDTO.class)@GetMapping("group")@ResponseBodypublic ResponseBean group() {log.info("demo--------group------");return ResponseBean.buildSuccess(userService.group());}}
2.1.2、测试
现在所有的User表数据汇总:
name | city | age |
---|---|---|
a | 北京 | 1 |
b | 北京 | 2 |
c | 山西 | 3 |
d | 山西 | 4 |
e | 河北 | 5 |
f | 江苏 | 6 |
2.1.2.1、访问测试
{"success": true,"result": [{"userId": null,"name": null,"city": "北京","status": null,"companyId": null,"companyName": null,"companyNameEnglish": null,"avgAge": 1,"sumAge": 3},{"userId": null,"name": null,"city": "山西","status": null,"companyId": null,"companyName": null,"companyNameEnglish": null,"avgAge": 3,"sumAge": 7},{"userId": null,"name": null,"city": "江苏","status": null,"companyId": null,"companyName": null,"companyNameEnglish": null,"avgAge": 6,"sumAge": 6},{"userId": null,"name": null,"city": "河北","status": null,"companyId": null,"companyName": null,"companyNameEnglish": null,"avgAge": 5,"sumAge": 5}],"msg": "","code": 200,"date": "1585563964870"
}
控制台日志
Logic SQL:
select city , sum(age) as sumAge, avg(age) as avgAge from user group by city Actual SQL: ds0 :::
select city , sum(age) as sumAge, avg(age) as avgAge , COUNT(age) AS AVG_DERIVED_COUNT_0 , SUM(age) AS AVG_DERIVED_SUM_0 from user_0 group by city ORDER BY city ASC ShardingSphere-SQL.log[89]Actual SQL: ds0 :::
select city , sum(age) as sumAge, avg(age) as avgAge , COUNT(age) AS AVG_DERIVED_COUNT_0 , SUM(age) AS AVG_DERIVED_SUM_0 from user_1 group by city ORDER BY city ASC Actual SQL: ds1 :::
select city , sum(age) as sumAge, avg(age) as avgAge , COUNT(age) AS AVG_DERIVED_COUNT_0 , SUM(age) AS AVG_DERIVED_SUM_0 from user_0 group by city ORDER BY city ASC Actual SQL: ds1 :::
select city , sum(age) as sumAge, avg(age) as avgAge , COUNT(age) AS AVG_DERIVED_COUNT_0 , SUM(age) AS AVG_DERIVED_SUM_0 from user_1 group by city ORDER BY city ASC
2.1.3、归纳总结:
成功,可以看到分表向每个数据库的每个表中发送了sql语句 (取出数据后,自动讲里面获取的数字进行计算)
2.3、between and
:成功
2.3.1、代码如下
2.3.1.1、UserService.java
public interface UserService {List<UserDTO> between();}
2.3.1.2、UserServiceImpl.java
@Service
@Slf4j
public class UserServiceImpl implements UserService {/*** between 成功*/@Overridepublic List<UserDTO> between() {Wrapper<User> userWrapper = new QueryWrapper<User>().lambda().between(User::getAge, 1, 3).orderByDesc(User::getCreateTime);List<User> users = userMapper.selectList(userWrapper);List<UserDTO> list = null;if (!EmptyUtil.isEmpty(users)) {list = users.stream().map(BeanUtils::userToDTO).collect(Collectors.toList());}return list;}}
2.3.1.3、UserController.java
@Controller
@RequestMapping("hlj/user")
@Slf4j
public class UserController {@ApiOperation(notes = "between",value = "group",consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,produces = MediaType.APPLICATION_JSON_VALUE,response = UserDTO.class)@GetMapping("between")@ResponseBodypublic ResponseBean between() {log.info("demo--------between------");return ResponseBean.buildSuccess(userService.between());}}
2.3.2、测试
现在所有的User表数据汇总:
name | city | age | createTime |
---|---|---|---|
a | 北京 | 1 | 2020-03-30 18:15:14 |
b | 北京 | 2 | 2020-03-30 18:16:14 |
c | 山西 | 3 | 2020-03-30 18:16:20 |
d | 山西 | 4 | 2020-03-30 18:16:27 |
e | 河北 | 5 | 2020-03-30 18:16:38 |
f | 江苏 | 6 | 2020-03-30 18:16:46 |
2.3.2.1、访问测试 :between 1 and 3 orderby createTime desc
{"success": true,"result": [{"id": "1244569141396078594","name": "c","city": "山西","age": 3,"status": "10","createTime": "2020-03-30 18:16:20","updateTime": "2020-03-30 18:16:20"},{"id": "1244569114653196290","name": "b","city": "北京","age": 2,"status": "10","createTime": "2020-03-30 18:16:14","updateTime": "2020-03-30 18:16:14"},{"id": "1244568862747492353","name": "a","city": "北京","age": 1,"status": "10","createTime": "2020-03-30 18:15:14","updateTime": "2020-03-30 18:15:14"}],"msg": "","code": 200,"date": "1585564394319"
}
2.3.3、归纳总结:
成功,可以看到分表向每个数据库的每个表中发送了sql语句
2.2、left join
: 连接查询
2.2.1、代码如下
2.2.1.1、UserMapper.java
public interface UserMapper extends BaseMapper<User> {List<UserRefCompany> leftJoin();}
2.2.1.2、UserMapper.xml
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd" >
<mapper namespace="com.healerjean.proj.dao.mapper.UserMapper"><select id="leftJoin" resultType="com.healerjean.proj.pojo.UserRefCompany">select u.id,u.name ,c.id as companyId,c.name as companyName,c.ref_user_id as refUserId,c.company_name_english as companyNameEnglishfrom user u left join company c on u.id = c .ref_user_id order by u.create_time</select></mapper>
2.2.1.3、UserService.java
@Service
@Slf4j
public class UserServiceImpl implements UserService {/*** left join 设置了绑定关系后成功*/@Overridepublic List<UserRefCompany> leftJoin() {List<UserRefCompany> userRefCompanies = userMapper.leftJoin();return userRefCompanies;}}
2.2.1.4、UserController.java
@Controller
@RequestMapping("hlj/user")
@Slf4j
public class UserController {@ApiOperation(notes = "leftJoin",value = "leftJoin",consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,produces = MediaType.APPLICATION_JSON_VALUE,response = UserDTO.class)@GetMapping("leftJoin")@ResponseBodypublic ResponseBean leftJoin() {log.info("demo--------leftJoin------");return ResponseBean.buildSuccess( userService.leftJoin());}}
2.2.2、访问测试:
现在所有的User表数据汇总:
id | name | city | age |
---|---|---|---|
1244568862747492353 | a | 北京 | 1 |
1244569114653196290 | b | 北京 | 2 |
1244569141396078594 | c | 山西 | 3 |
1244569167870525442 | d | 山西 | 4 |
1244569214813175810 | e | 河北 | 5 |
1244569246752800769 | f | 江苏 | 6 |
现在所有的company
表数据汇总、添加这个数据的时候,其实我们已经做了一些小动作了,company表的分库分表是按照ref_user_id进行分库的(和user表中id的分库策略一致),这样就保证了后面绑定关系之后查询的成功
id | ref_user_id | name | company_name_english | create_time |
---|---|---|---|---|
1244575632182181889 | 1244568862747492353 | 北京小米有限公司 | xiaomi | 2020-03-30 18:42:08 |
1244575692508856321 | 1244569114653196290 | 北京字节跳动 | zijie | 2020-03-30 18:42:22 |
1244575754915905538 | 1244569141396078594 | 山西爱酷科技 | Iku | 2020-03-30 18:42:37 |
1244575817167765506 | 1244569214813175810 | 河北腾讯 | tecent | 2020-03-30 18:42:52 |
2.2.2.1、默认失败
{"success": true,"result": [{"userId": null,"name": "a","city": null,"status": null,"companyId": 1244585691742093300,"companyName": "北京小米有限公司","companyNameEnglish": "xiaomi","avgAge": null,"sumAge": null},{"userId": null,"name": "a","city": null,"status": null,"companyId": null,"companyName": null,"companyNameEnglish": null,"avgAge": null,"sumAge": null},{"userId": null,"name": "b","city": null,"status": null,"companyId": 1244585869291176000,"companyName": "北京字节跳动","companyNameEnglish": "zijie","avgAge": null,"sumAge": null},{"userId": null,"name": "b","city": null,"status": null,"companyId": null,"companyName": null,"companyNameEnglish": null,"avgAge": null,"sumAge": null},{"userId": null,"name": "c","city": null,"status": null,"companyId": 1244585918465196000,"companyName": "山西爱酷科技","companyNameEnglish": "Iku","avgAge": null,"sumAge": null},{"userId": null,"name": "c","city": null,"status": null,"companyId": null,"companyName": null,"companyNameEnglish": null,"avgAge": null,"sumAge": null},{"userId": null,"name": "d","city": null,"status": null,"companyId": null,"companyName": null,"companyNameEnglish": null,"avgAge": null,"sumAge": null},{"userId": null,"name": "d","city": null,"status": null,"companyId": null,"companyName": null,"companyNameEnglish": null,"avgAge": null,"sumAge": null},{"userId": null,"name": "e","city": null,"status": null,"companyId": 1244585970768167000,"companyName": "河北腾讯","companyNameEnglish": "tecent","avgAge": null,"sumAge": null},{"userId": null,"name": "e","city": null,"status": null,"companyId": null,"companyName": null,"companyNameEnglish": null,"avgAge": null,"sumAge": null},{"userId": null,"name": "f","city": null,"status": null,"companyId": null,"companyName": null,"companyNameEnglish": null,"avgAge": null,"sumAge": null},{"userId": null,"name": "f","city": null,"status": null,"companyId": null,"companyName": null,"companyNameEnglish": null,"avgAge": null,"sumAge": null}],"msg": "","code": 200,"date": "1585567537634"
}
Response Code
控制台日志(整理了下),可以看到下面的查询类似于卡迪尔积,每个x和每个y数组里面的分别join了一次,所以导致了错误的出现,
要查询的结果
Logic SQL: select u.id,u.name ,c.id as companyId,c.name as companyName,c.ref_user_id as refUserId,c.company_name_english as companyNameEnglishfrom user uleft join company c on u.id = c .ref_user_id ShardingSphere-SQL.log[89]分库查询
Actual SQL: ds0 :::
select ^^^ from user_1 u left join company_1 c on u.id = c .ref_user_id
select ^^^ from user_1 u left join company_0 c on u.id = c .ref_user_id
select ^^^ from user_0 u left join company_1 c on u.id = c .ref_user_id
select ^^^ from user_0 u left join company_0 c on u.id = c .ref_user_id Actual SQL: ds1 :::
select ^^^ from user_1 u left join company_1 c on u.id = c .ref_user_id
select ^^^ from user_1 u left join company_0 c on u.id = c .ref_user_id
select ^^^ from user_0 u left join company_1 c on u.id = c .ref_user_id
select ^^^ from user_0 u left join company_0 c on u.id = c .ref_user_id
2.2.3、配置user和company的绑定关系 :成功
如果要保证一对一的表关系,必须保证2点
1、user和company的表数量以及所在节点要完全一模一样
2、从表的的外键规则和主表的外键分库分表策略要一直
# user company 分表
# user_0,user_1,user_2(自定义分表算法)
spring.shardingsphere.sharding.tables.user.actual-data-nodes=ds$->{0..1}.user_$->{0..1}
spring.shardingsphere.sharding.tables.user.table-strategy.standard.sharding-column=id
spring.shardingsphere.sharding.tables.user.table-strategy.standard.precise-algorithm-class-name=com.healerjean.proj.config.datasource.CustomShardingTableAlgorithm
# company_0,company_1 (inline分表策略 表达式 id%2)
spring.shardingsphere.sharding.tables.company.actual-data-nodes=ds$->{0..1}.company_$->{0..1}
spring.shardingsphere.sharding.tables.company.database-strategy.inline.sharding-column=ref_user_id
spring.shardingsphere.sharding.tables.company.database-strategy.inline.algorithm-expression=ds${ref_user_id.longValue() % 2}
spring.shardingsphere.sharding.tables.company.table-strategy.inline.sharding-column=ref_user_id
spring.shardingsphere.sharding.tables.company.table-strategy.inline.algorithm-expression=company_${ref_user_id.longValue() % 2}# 绑定表规则列表
spring.shardingsphere.sharding.binding-tables[0]=user,company
2.2.3.1、访问测试
{"success": true,"result": [{"userId": null,"name": "a","city": null,"status": null,"companyId": 1244585691742093300,"companyName": "北京小米有限公司","companyNameEnglish": "xiaomi","avgAge": null,"sumAge": null},{"userId": null,"name": "b","city": null,"status": null,"companyId": 1244585869291176000,"companyName": "北京字节跳动","companyNameEnglish": "zijie","avgAge": null,"sumAge": null},{"userId": null,"name": "c","city": null,"status": null,"companyId": 1244585918465196000,"companyName": "山西爱酷科技","companyNameEnglish": "Iku","avgAge": null,"sumAge": null},{"userId": null,"name": "d","city": null,"status": null,"companyId": null,"companyName": null,"companyNameEnglish": null,"avgAge": null,"sumAge": null},{"userId": null,"name": "e","city": null,"status": null,"companyId": 1244585970768167000,"companyName": "河北腾讯","companyNameEnglish": "tecent","avgAge": null,"sumAge": null},{"userId": null,"name": "f","city": null,"status": null,"companyId": null,"companyName": null,"companyNameEnglish": null,"avgAge": null,"sumAge": null}],"msg": "","code": 200,"date": "1585567592799"
}
控制台日志
要查询的结果
Logic SQL: select u.id,u.name ,c.id as companyId,c.name as companyName,c.ref_user_id as refUserId,c.company_name_english as companyNameEnglishfrom user uleft join company c on u.id = c .ref_user_id ShardingSphere-SQL.log[89]分库查询
Actual SQL: ds0 :::
select ^^^ from user_1 u left join company_1 c on u.id = c .ref_user_id
select ^^^ from user_0 u left join company_0 c on u.id = c .ref_user_id Actual SQL: ds1 :::
select ^^^ from user_1 u left join company_1 c on u.id = c .ref_user_id
select ^^^ from user_0 u left join company_0 c on u.id = c .ref_user_id
2.3、默认数据源,配合数据源看
默认数据源指定(不分库的表)
默认数据源指定(不分库的表)
spring.shardingsphere.sharding.default-data-source-name=ds0
2.4、配置广播表
为了防止跨库查,可以将那些永远不会改变的表。在每个库中复制一份。一般适用于配置类的数据,(如果插入数据,会向所有的数据库同时进行插入)
# 配置广播表(为了防止跨库查,可以将那些永远不会改变的表。在每个库中复制一份。一般适用于配置类的数据,(如果插入数据,会向所有的数据库同时进行插入)
spring.shardingsphere.sharding.broadcast-tables=demo_entity
分库分表之_分库分表 + 复杂查询相关推荐
- mysql 分表 好处_分库分表浅谈
什么是分库分表 顾名思义,分库分表就是按照一定的规则,对原有的数据库和表进行拆分,把原本存储于一个库的数据分块存储到多个库上,把原本存储于一个表的数据分块存储到多个表上. 为什么需要分库分表 随着 ...
- php mysql oracle数据库表结构图_创建数据库表
数据库的作用:1.有结构的存储大量数据.2.有效保持数据的一致性.3.方便智能的分析,产生新的有用的信息.4.满足应用的共享和安全的要求. 关系型数据库的基本组成:一个数据库是由一组数据表(table ...
- 用java写注册表单_利用HTML表单标签编写一个注册页面
今天我们来写一个注册页面 form表单 先来利用表单标签制作一个简单的注册页面,给大家说说标签的结构: 页面结构大体就是这样子的~ 利用HTML表单标签编写一个注册页面 表单标签: 所有需要提交到服务 ...
- 增大mysql修改表空间_扩充数据库表空间
ALTER TABLESPACE ADD DATAFILE , [REUSE] NEXT MAXSIZE <>中是你要填的内容,有|是选其一. 如:增加文件是d:\dbfs\mydatab ...
- mysql表前缀_关于数据库表前缀的认识
mysql数据库表前缀,这个是我们区分其它表的一个方式,当我们同一个数据库中含有多个系统的的时候,表前缀就却分的唯一标识.我们使用php开源程序安装建站的时候,一般数据库表前缀都是默认设置好的,如:w ...
- jpi多表联查_数据库两表联查、多表联查,多重联查
表连接查询 [小编用的是Oracle数据库,Oracle数据库区分表名与字段名的大小写,所以大家进行查询的时候记得注意双引号哟~] 有表 表名:AAA 字段: id name tag 表名:BBB 字 ...
- 广义表取表头表尾_数据结构广义表的递归算法
Hello同学们,又到了美妙的星期三,很开心又和大家见面了.这次我们要来讲一讲关于广义表的那些事儿! 俗语说:"与其临渊羡鱼,不如退而结网." 希望通过今天的学习大家可以有所收获. ...
- mysql from多表顺序_数据库 from 表的顺序
MY SQL语句常用集合 1个数据库通常包含一个或多个表.每个表由一个名字标识 1.SELECT - 从数据库表中获取数据 UPDATE - 更新数据库表中的数据 DELETE - 从数据库表中删除数 ...
- mysql 用户签到表设计_用户签到表的设计思路与数据库实现
签到做为,一个促进用户粘性的手段已经很成熟了. 这里说下,个人的一些设计表思路 添加用户配置表.主键关联User表,即可. CREATE TABLE `cft_user_signin` ( `id` ...
最新文章
- 20145223 《信息安全系统设计基础》课程总结
- 关于A*寻路算法的认识
- python 入门DAY1
- FileNotFoundError: Could not find module 'xxx.dll'. Try using the full path with constructor syntax.
- 即插即用的轻量注意力机制ECA--Net
- MFC显示JPG,bmp图片
- Spring框架Runtime介绍(导包)
- DHCP的安装到简单测试(tar方式)
- cookie 百科_Cookie和session应该这样理解
- C程序设计语言现代方法15:编写大型程序
- 《MYSQL必知必会》—1.了解SQL
- 完全掌握1级日本与能力考试语法问题对策
- 产品沉思录 V3.0 试读
- linux系统下载7.0,redhat7.0_redhat enterprise linux 7.0下载 附安装教程 - 121下载站
- ATAPI(磁盘端口驱动)级文件保护简单实现
- 《赐我》-一只白羊 同步歌词
- 服务器摆放需要预留U位么_办公沙发摆放有何讲究?
- VCN 在windows和linux之间 复制粘贴
- Ubuntu中如何开启samba/smb共享
- spring-注解实现自动装配