2022-10-09 mysql列存储引擎-exists结果错误-问题分析
2024-05-20 13:01:24
摘要:
TPCH的Q4使用了exists子查询, 但是结果与innodb有出入, 本文分析原因。
DDL:
selecto_orderpriority,count(*) as order_countfromorderswhereo_orderdate >= date '1993-07-01'and o_orderdate < date '1993-07-01' + interval '3' monthand exists (select*fromlineitemwherel_orderkey = o_orderkeyand l_commitdate < l_receiptdate)group byo_orderpriorityorder byo_orderprioritylimit 10;查询结果对比：
问题分析:
mysql列存储引擎执行过程分析:
到TMD的语法树:
T:-1 = TABLE_ALIAS(T:0,"orders")
T:-2 = TMP_TABLE(T:4294967295)VC:-2.0 = CREATE_VC(T:-2,PHYS_COL(T:-1,A:5))
A:-1 = T:-2.ADD_COLUMN(VC:-2.0,GROUP_BY,"o_orderpriority"
A:-2 = T:-2.ADD_COLUMN(<null>,COUNT,"order_count","ALL")
VC:-2.1 = CREATE_VC(T:-2,PHYS_COL(T:-2,A:-1))
T:-2.ADD_ORDER(VC:-2.1,ASC)
VC:-2.2 = CREATE_VC(T:-2,PHYS_COL(T:-1,A:4))
VC:-2.3 = CREATE_VC(T:-2,EXPR("date_literal"))
C:0 = CREATE_CONDS(T:-2,VC:-2.2,>=,VC:-2.3,<null>)
VC:-2.4 = CREATE_VC(T:-2,EXPR("date_add_interval"))
C:0.AND(VC:-2.2,<,VC:-2.4,<null>)T:-3 = TABLE_ALIAS(T:1,"lineitem")
T:-4 = TMP_TABLE(T:4294967293)VC:-4.0 = CREATE_VC(T:-4,EXPR("1"))
A:-1 = T:-4.ADD_COLUMN(VC:-4.0,LIST,"Not_used","ALL")
VC:-4.1 = CREATE_VC(T:-4,PHYS_COL(T:-3,A:0))
VC:-4.2 = CREATE_VC(T:-4,EXPR("TIANMU_FIELD(T:-1,A:0)"))
C:1 = CREATE_CONDS(T:-4,VC:-4.1,=,VC:-4.2,<null>)
VC:-4.3 = CREATE_VC(T:-4,PHYS_COL(T:-3,A:11))
VC:-4.4 = CREATE_VC(T:-4,PHYS_COL(T:-3,A:12))
C:1.AND(VC:-4.3,<,VC:-4.4,<null>)
T:-4.ADD_CONDS(C:1,WHERE)
T:-4.APPLY_CONDS()
T:-4.MODE(LIMIT,0,1)VC:-2.5 = CREATE_VC(T:-2,SUBQUERY(T:-4))
C:0.AND(VC:-2.5,EXISTS,<null>,<null>)
T:-2.ADD_CONDS(C:0,WHERE)T:-2.APPLY_CONDS()
T:-2.MODE(LIMIT,0,10)
RESULT(T:-2)关键点:
T:-3 = TABLE_ALIAS(T:1,"lineitem")
T:-4 = TMP_TABLE(T:4294967293)VC:-4.0 = CREATE_VC(T:-4,EXPR("1"))
A:-1 = T:-4.ADD_COLUMN(VC:-4.0,LIST,"Not_used","ALL")
VC:-4.1 = CREATE_VC(T:-4,PHYS_COL(T:-3,A:0))
VC:-4.2 = CREATE_VC(T:-4,EXPR("TIANMU_FIELD(T:-1,A:0)"))
C:1 = CREATE_CONDS(T:-4,VC:-4.1,=,VC:-4.2,<null>)
VC:-4.3 = CREATE_VC(T:-4,PHYS_COL(T:-3,A:11))
VC:-4.4 = CREATE_VC(T:-4,PHYS_COL(T:-3,A:12))
C:1.AND(VC:-4.3,<,VC:-4.4,<null>)
T:-4.ADD_CONDS(C:1,WHERE)
T:-4.APPLY_CONDS()
T:-4.MODE(LIMIT,0,1)
VC:-4.1 = CREATE_VC(T:-4,PHYS_COL(T:-3,A:0))
VC:-4.2 = CREATE_VC(T:-4,EXPR("TIANMU_FIELD(T:-1,A:0)"))
C:1 = CREATE_CONDS(T:-4,VC:-4.1,=,VC:-4.2,<null>)分析:
- TMD引擎的子查询, 比较的o_orderkey, 不是外部表orders中传递过来的
- 而是在子查询时, 对lineitem和orders进行关联查询做出的
- 等价于以下mysql/innodb的处理
mysql> select-> o_orderpriority,-> count(*) as order_count-> from-> orders-> where-> o_orderdate >= date '1993-07-01'-> and o_orderdate < date '1993-07-01' + interval '3' month-> and exists (-> select-> *-> from-> lineitem,-> orders-> where-> l_orderkey = o_orderkey-> and l_commitdate < l_receiptdate-> )-> group by-> o_orderpriority-> order by-> o_orderpriority-> limit 10;
+-----------------+-------------+
| o_orderpriority | order_count |
+-----------------+-------------+
| 1-URGENT | 1099 |
| 2-HIGH | 1085 |
| 3-MEDIUM | 1102 |
| 4-NOT SPECIFIED | 1075 |
| 5-LOW | 1191 |
+-----------------+-------------+
5 rows in set (0.03 sec)
mysql列存储引擎对于查询计划, 等价于
一. 精简后的SQL
selecto_orderpriorityfromorderswhereo_orderdate >= date '1993-07-01'and o_orderdate < date '1993-07-01' + interval '3' monthand exists (select*fromlineitemwherel_orderkey = o_orderkeyand l_commitdate < l_receiptdate)limit 10;
二. 查询计划可以理解为
selecto_orderpriorityfromorderswhereo_orderdate >= date '1993-07-01'and o_orderdate < date '1993-07-01' + interval '3' monthand count (select1fromorders,lineitemwherel_orderkey = o_orderkeyand l_commitdate < l_receiptdatelimit 1) == 1limit 10;trace显示:
mysql> select * from INFORMATION_SCHEMA.OPTIMIZER_TRACE\G
*************************** 1. row ***************************QUERY: selecto_orderpriority,count(*) as order_countfromorderswhereo_orderdate >= date '1993-07-01'and o_orderdate < date '1993-07-01' + interval '3' monthand exists (select*fromlineitemwherel_orderkey = o_orderkeyand l_commitdate < l_receiptdate)group byo_orderpriorityorder byo_orderprioritylimit 10TRACE: {"steps": [{"join_preparation": {"select#": 1,"steps": [{"join_preparation": {"select#": 2,"steps": [{"expanded_query": "/* select#2 */ select 1 from `lineitem` where ((`lineitem`.`l_orderkey` = `orders`.`o_orderkey`) and (`lineitem`.`l_commitdate` < `lineitem`.`l_receiptdate`))"}]}},{"expanded_query": "/* select#1 */ select `orders`.`o_orderpriority` AS `o_orderpriority`,count(0) AS `order_count` from `orders` where ((`orders`.`o_orderdate` >= DATE'1993-07-01') and (`orders`.`o_orderdate` < (DATE'1993-07-01' + interval '3' month)) and exists(/* select#2 */ select 1 from `lineitem` where ((`lineitem`.`l_orderkey` = `orders`.`o_orderkey`) and (`lineitem`.`l_commitdate` < `lineitem`.`l_receiptdate`)))) group by `orders`.`o_orderpriority` order by `orders`.`o_orderpriority` limit 10"}]}},{"join_optimization": {"select#": 1,"steps": [{"condition_processing": {"condition": "WHERE","original_condition": "((`orders`.`o_orderdate` >= DATE'1993-07-01') and (`orders`.`o_orderdate` < (DATE'1993-07-01' + interval '3' month)) and exists(/* select#2 */ select 1 from `lineitem` where ((`lineitem`.`l_orderkey` = `orders`.`o_orderkey`) and (`lineitem`.`l_commitdate` < `lineitem`.`l_receiptdate`))))","steps": [{"transformation": "equality_propagation","subselect_evaluation": [],"resulting_condition": "((`orders`.`o_orderdate` >= DATE'1993-07-01') and (`orders`.`o_orderdate` < (DATE'1993-07-01' + interval '3' month)) and exists(/* select#2 */ select 1 from `lineitem` where ((`lineitem`.`l_orderkey` = `orders`.`o_orderkey`) and (`lineitem`.`l_commitdate` < `lineitem`.`l_receiptdate`))))"},{"transformation": "constant_propagation","subselect_evaluation": [],"resulting_condition": "((`orders`.`o_orderdate` >= DATE'1993-07-01') and (`orders`.`o_orderdate` < (DATE'1993-07-01' + interval '3' month)) and exists(/* select#2 */ select 1 from `lineitem` where ((`lineitem`.`l_orderkey` = `orders`.`o_orderkey`) and (`lineitem`.`l_commitdate` < `lineitem`.`l_receiptdate`))))"},{"transformation": "trivial_condition_removal","subselect_evaluation": [],"resulting_condition": "((`orders`.`o_orderdate` >= DATE'1993-07-01') and (`orders`.`o_orderdate` < (DATE'1993-07-01' + interval '3' month)) and exists(/* select#2 */ select 1 from `lineitem` where ((`lineitem`.`l_orderkey` = `orders`.`o_orderkey`) and (`lineitem`.`l_commitdate` < `lineitem`.`l_receiptdate`))))"}]}}]}}]
}
MISSING_BYTES_BEYOND_MAX_MEM_SIZE: 0INSUFFICIENT_PRIVILEGES: 0
1 row in set (0.00 sec)
*************************** 1. row ***************************id: 1select_type: PRIMARYtable: orderspartitions: NULLtype: ALL
possible_keys: NULLkey: NULLkey_len: NULLref: NULLrows: 150000filtered: 11.11Extra: Using where with pushed condition ((`tpch`.`orders`.`o_orderdate` >= DATE'1993-07-01') and (`tpch`.`orders`.`o_orderdate` < <cache>((DATE'1993-07-01' + interval '3' month))) and exists(/* select#2 */ select 1 from `tpch`.`lineitem` where ((`tpch`.`lineitem`.`l_orderkey` = `tpch`.`orders`.`o_orderkey`) and (`tpch`.`lineitem`.`l_commitdate` < `tpch`.`lineitem`.`l_receiptdate`))))(t0) Pckrows: 3, susp. 3 (0 empty 0 full). Conditions: 2; Using temporary; Using filesort
*************************** 2. row ***************************id: 2select_type: DEPENDENT SUBQUERYtable: lineitempartitions: NULLtype: ref
possible_keys: PRIMARYkey: PRIMARYkey_len: 4ref: tpch.orders.o_orderkeyrows: 60058filtered: 33.33Extra: Using where with pushed condition (`tpch`.`lineitem`.`l_commitdate` < `tpch`.`lineitem`.`l_receiptdate`)(t0) Pckrows: 10, susp. 10 (0 empty 0 full). Conditions: 1
2 rows in set, 2 warnings (0.06 sec)
核对exists传值的问题点
一. 对比exists传值
select*fromorderswhereo_orderdate >= date '1993-07-01'and o_orderdate < date '1993-07-01' + interval '3' monthand exists (select 641 as o_orderkey)order byo_totalpricelimit 10;
+------------+-----------+---------------+--------------+-------------+-----------------+-----------------+----------------+----------------------------------------------------------------------------+
| o_orderkey | o_custkey | o_orderstatus | o_totalprice | o_orderdate | o_orderpriority | o_clerk | o_shippriority | o_comment |
+------------+-----------+---------------+--------------+-------------+-----------------+-----------------+----------------+----------------------------------------------------------------------------+
| 591877 | 11008 | F | 921.74 | 1993-09-27 | 4-NOT SPECIFIED | Clerk#000000141 | 0 | haggle ruthlessly against the carefully idle deposits. fluffily pending t |
| 339815 | 13199 | F | 924.63 | 1993-09-05 | 5-LOW | Clerk#000000993 | 0 | s accounts along the bold, final deposits c |
| 106983 | 5350 | F | 1037.92 | 1993-09-14 | 2-HIGH | Clerk#000000474 | 0 | te blithely. blithely ironic deposits about the furiou |
| 522276 | 4183 | F | 1051.16 | 1993-09-13 | 1-URGENT | Clerk#000000803 | 0 | theodolites. blithe |
| 524641 | 8362 | F | 1069.35 | 1993-09-12 | 3-MEDIUM | Clerk#000000198 | 0 | . accounts haggle blithely above the furiou |
| 510020 | 8209 | F | 1228.11 | 1993-09-19 | 4-NOT SPECIFIED | Clerk#000000080 | 0 | ironic, even pinto beans run carefully. quickly ironic dependencie |
| 34338 | 6896 | F | 1265.76 | 1993-07-01 | 5-LOW | Clerk#000000242 | 0 | old deposits affix ironic |
| 363879 | 13861 | F | 1267.84 | 1993-08-09 | 2-HIGH | Clerk#000000685 | 0 | tes wake boldly blithe packages. furiously final platelets sleep s |
| 398497 | 10006 | F | 1284.22 | 1993-07-07 | 1-URGENT | Clerk#000000592 | 0 | x-ray. express platelets above the bold platelets are fluffily fi |
| 435299 | 3914 | F | 1305.68 | 1993-07-17 | 2-HIGH | Clerk#000000830 | 0 | ccounts cajole. carefully ironic requests sleep sp |
+------------+-----------+---------------+--------------+-------------+-----------------+-----------------+----------------+----------------------------------------------------------------------------+
10 rows in set (0.05 sec)
selecto_orderkeyfromorderswhereo_orderdate >= date '1993-07-01'and o_orderdate < date '1993-07-01' + interval '3' monthand exists (selecto_orderkeyfromlineitem,orderswherel_orderkey = o_orderkeyand l_commitdate < l_receiptdateand o_orderdate < date '1993-07-01' + interval '3' monthand o_orderdate >= date '1993-07-01'and l_orderkey = 36673limit 1)order byo_totalpricelimit 10;Empty set (0.18 sec)
selecto_orderkeyfromlineitem,orderswherel_orderkey = o_orderkeyand l_commitdate < l_receiptdateand o_orderdate < date '1993-07-01' + interval '3' monthand o_orderdate >= date '1993-07-01'and l_orderkey = 36673limit 1;+------------+
| o_orderkey |
+------------+
| 36673 |
+------------+
1 row in set (0.02 sec)
2022-10-09 mysql列存储引擎-exists结果错误-问题分析相关推荐
- 2022-09-20 mysql列存储引擎-问题定位-去除索引逻辑后引发crash
摘要: mysql列存储引擎,此前的索引逻辑功能不完备.去除对索引逻辑的使用后,引发crash. 本文分析其原因. 去除索引逻辑的做法: case JT_EQ_REF:{TABLE* table = ...
- 2022-12-08 mysql列存储引擎-POC-CQ慢SQL4-上下文记录
摘要: mysql列存储引擎-CQ慢SQL4-上下文记录 关联: https://stoneatom.yuque.com/staff-ft8n1u/qfqtnb/cy57cc0ecm81fyp1 关联 ...
- 2022-12-16 mysql列存储引擎-YP-上下文记录
摘要: mysql列存储引擎-一铺-上下文记录 DDL: 表结构: create table tunit(id int not null,unitsn varchar(20) not null );c ...
- 2022-09-15 mysql列存储引擎-语法树转换
摘要: 列存储引擎有一套自己的执行处理规则, 在进行处理前,是将mysql经过词法分析和语法分析后的语法树,经过了一些符合自己逻辑的处理. 本文记录mysql的语法树在列存储引擎中的转换过程. 逻辑建 ...
- MySQL的存储引擎InnoDB选择了B+ 树
我们知道数据的存储和检索是两个很重要的功能,当我们的数据量大了,怎么能快速的检索数据呢,答案是使用索引,可索引具体的技术实现有很多,选择哪一种呢,我就以mysql为例记录下它为什么选择了B+树作为索引 ...
- MySQL数据库03(MySQL的存储引擎 DML语句 父子查询 )
一.1.MySQL的存储引擎 储存引擎的类型:MyISAM.InnoDB. Memory.CSV等九种 MyiSAM和InnoDB类型主要区别 名称 InnoDB MyISAM 事务处理 支持 不支持 ...
- MySQL—05—MySQL如何处理SQL语句;MySQL数据库存储引擎介绍;
一. MySQL 中的执行计划 1 MySQL 执行计划 在 MySQL 中可以通过 explain 关键字模拟优化器,执行 SQL 语句,从而知道 MySQL 是 如何处理 SQL 语句的. 2 M ...
- MySQL常用存储引擎之Innodb
在MYSQL5.5版本之后,具体是在5.58版本之后,InnoDB代替MYISAM称为MYSQL的默认存储引擎,说实话呢,是一个非常重要的事情,之前在使用MYSQL时,说别人使用InnoDB,而不用m ...
- MySql 扩展存储引擎
MySql 扩展存储引擎 下面介绍几个列式存储引擎(都有两个版本:社区版.商业版): 一:TokuDB TokuDB 是一个高性能.支持事务处理的 MySQL 和 MariaDB 的存储引擎.Toku ...
最新文章
- UE4创建第一人称射击游戏学习教程
- svpwm矢量控制电机相电压波形_如何深入理解SVPWM？
- express利用nodemailer发送邮件（163邮箱）
- python wx提示框字体_使用wxStyledTextCtrl实现代码提示
- 不到 100 行 Python 代码徐峥变葛优
- BT5 R1不能启动ibus输入法解决方案
- Windows Server 2016补丁更新机制
- CGI、FastCGI和php-fpm的概念和区别
- OPPO 物联网开放之路
- pluto.ctl_Apache Pluto，Portlet Bridge和JSF 2.0集成示例教程
- 人生五大投资，你投对了几个？
- Lync 2013持久聊天迁移至Skype for Business
- 记账系统推荐金蝶精斗云_小编总结了金蝶精斗云财务软件的优劣势
- 超低功耗高性能2.4GHz GFSK 无线收发芯片Si24R1替代NRF24L01P
- 【从嵌入式视角学习香山处理器】一、如何开始？（开发环境搭建）
- 稳压二极管的工作原理是什么？
- 小白一键重装linux系统重装,小白一键重装系统官网
- 做不到想做的，真难受~
- 构造Linux的图形化安装程序（4）(转)
- sql Server2005 master损坏处理