MonetDB heap bug?
2024-05-15 19:12:56
接前面的测试 :
http://blog.163.com/digoal@126/blog/static/1638770402014714104326879/
本文用到的2个测试表, 分别有60个字段, 其中1个字段为INT, 全唯一, 其他varchar(64), 全随机.
其中一个表b 5000万记录, 另一个表c 2500万记录.
数据库总共占用215GB.
数据库服务器内存+SWAP一共97GB.
但是执行某些查询的时候, 数据库目录会暴增到1.1TB, 例如
不带INT类型的分组查询.
某些关联则需要更多空间, 甚至导致空间爆满. 报错.
sql>select count(*) from (select c2,c11,c21,c31,c41,c51,c60 from b group by c2,c11,c21,c31,c41,c51,c60) as t;
+----------+
| L1 |
+==========+
| 50000000 |
+----------+
1 tuple (10m 21s)某些关联则需要更多空间, 甚至导致空间爆满. 报错.
如下 :
sql>select count(*) from b,c where b.c4=c.c5 and b.c2=c.c3;
+------+
| L1 |
+======+
| 0 |
+------+
1 tuple (31.8s)
sql>select count(*) from b,c where b.c4=c.c5 or b.c2=c.c3;
GDK reported error.
HEAPalloc: Insufficient space for HEAP of 10000000000000000 bytes.
HEAPalloc: Insufficient space for HEAP of 10000000000000000 bytes.
sql>select count(*) from b,c where b.c4=c.c5 or b.c2=c.c3;
could not allocate space for
HEAPextend: failed to extend to 115215433728 for 54/5473.theap: h->storage == STORE_MEM && can_map && fd >= 0 && HEAPload_intern() < 0
存储信息, 注意column size和heap size.
int类型heapsize=0
目前尚不清楚是不是MonetDB的BUG, 看起来不适合字符串这样的变长类型.
sql>select * from storage where "table" in('b','c');
+--------+-------+-----------+---------+----------+----------+-----------+------------+------------+---------+--------+
| schema | table | column | type | location | count | typewidth | columnsize | heapsize | indices | sorted |
+========+=======+===========+=========+==========+==========+===========+============+============+=========+========+
| sys | b | c1 | int | 10/1070 | 50000000 | 4 | 200000000 | 0 | 0 | false |
| sys | b | c2 | varchar | 10/1072 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c3 | varchar | 11/1100 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c4 | varchar | 10/1073 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c5 | varchar | 10/1071 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c6 | varchar | 10/1076 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c7 | varchar | 10/1074 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c8 | varchar | 14/1443 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c9 | varchar | 14/1444 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c10 | varchar | 14/1445 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c11 | varchar | 14/1446 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c12 | varchar | 14/1447 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c13 | varchar | 14/1450 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c14 | varchar | 14/1451 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c15 | varchar | 14/1452 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c16 | varchar | 14/1453 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c17 | varchar | 14/1454 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c18 | varchar | 14/1455 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c19 | varchar | 14/1456 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c20 | varchar | 14/1457 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c21 | varchar | 14/1460 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c22 | varchar | 14/1461 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c23 | varchar | 14/1462 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c24 | varchar | 14/1463 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c25 | varchar | 14/1464 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c26 | varchar | 14/1465 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c27 | varchar | 14/1466 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c28 | varchar | 14/1467 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c29 | varchar | 14/1470 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c30 | varchar | 14/1471 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c31 | varchar | 14/1472 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c32 | varchar | 14/1473 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c33 | varchar | 14/1474 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c34 | varchar | 14/1475 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c35 | varchar | 14/1476 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c36 | varchar | 14/1477 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c37 | varchar | 15/1500 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c38 | varchar | 15/1501 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c39 | varchar | 15/1502 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c40 | varchar | 15/1503 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c41 | varchar | 15/1504 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c42 | varchar | 15/1505 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c43 | varchar | 15/1506 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c44 | varchar | 15/1507 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c45 | varchar | 15/1510 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c46 | varchar | 15/1511 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c47 | varchar | 15/1512 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c48 | varchar | 15/1513 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c49 | varchar | 15/1514 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c50 | varchar | 15/1515 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c51 | varchar | 15/1516 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c52 | varchar | 15/1517 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c53 | varchar | 15/1520 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c54 | varchar | 15/1521 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c55 | varchar | 15/1522 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c56 | varchar | 15/1523 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c57 | varchar | 15/1524 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c58 | varchar | 15/1525 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c59 | varchar | 15/1526 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | c60 | varchar | 15/1527 | 50000000 | 34 | 200000000 | 2400321536 | 0 | false |
| sys | b | b_c1_pkey | oid | 13/1316 | 0 | 8 | 0 | 0 | 0 | true |
| sys | c | c1 | int | 10/1035 | 25000000 | 4 | 100000000 | 0 | 0 | true |
| sys | c | c2 | varchar | 06/675 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c3 | varchar | 03/370 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c4 | varchar | 27/2774 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c5 | varchar | 27/2775 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c6 | varchar | 27/2776 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c7 | varchar | 27/2777 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c8 | varchar | 30/3000 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c9 | varchar | 30/3001 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c10 | varchar | 30/3002 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c11 | varchar | 30/3003 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c12 | varchar | 30/3004 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c13 | varchar | 30/3005 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c14 | varchar | 30/3006 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c15 | varchar | 30/3007 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c16 | varchar | 30/3010 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c17 | varchar | 30/3011 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c18 | varchar | 30/3012 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c19 | varchar | 30/3013 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c20 | varchar | 30/3014 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c21 | varchar | 30/3015 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c22 | varchar | 30/3016 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c23 | varchar | 30/3017 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c24 | varchar | 30/3020 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c25 | varchar | 30/3021 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c26 | varchar | 30/3022 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c27 | varchar | 30/3023 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c28 | varchar | 30/3024 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c29 | varchar | 30/3025 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c30 | varchar | 30/3026 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c31 | varchar | 30/3027 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c32 | varchar | 30/3030 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c33 | varchar | 30/3031 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c34 | varchar | 30/3032 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c35 | varchar | 30/3033 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c36 | varchar | 30/3034 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c37 | varchar | 30/3035 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c38 | varchar | 30/3036 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c39 | varchar | 30/3037 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c40 | varchar | 30/3040 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c41 | varchar | 30/3041 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c42 | varchar | 30/3042 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c43 | varchar | 30/3043 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c44 | varchar | 30/3044 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c45 | varchar | 30/3045 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c46 | varchar | 30/3046 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c47 | varchar | 30/3047 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c48 | varchar | 30/3050 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c49 | varchar | 30/3051 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c50 | varchar | 30/3052 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c51 | varchar | 30/3053 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c52 | varchar | 30/3054 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c53 | varchar | 30/3055 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c54 | varchar | 30/3056 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c55 | varchar | 30/3057 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c56 | varchar | 30/3060 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c57 | varchar | 30/3061 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c58 | varchar | 30/3062 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c59 | varchar | 30/3063 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
| sys | c | c60 | varchar | 30/3064 | 25000000 | 34 | 100000000 | 1200160768 | 0 | false |
+--------+-------+-----------+---------+----------+----------+-----------+------------+------------+---------+--------+
121 tuples (7.312ms)目前尚不清楚是不是MonetDB的BUG, 看起来不适合字符串这样的变长类型.
需要了解详情需对MonetDB代码有一定了解.
如果要使用的话, 建议将字符串类型字典话, 使用固定长度的类型, 例如使用INT类型或者INT8类型替代.
后面会测试一下int类型的性能.
MonetDB heap bug?相关推荐
- 一文搞懂栈(stack)、堆(heap)、单片机裸机内存管理malloc
大家好,我是无际. 有一周没水文了,俗话说夜路走多了难免遇到鬼. 最近就被一个热心网友喷了. 说我的文章没啥营养,所以今天来一篇烧脑的. 哈哈,开个玩笑,不要脸就没人能把我绑架. 主要是最近研发第二代 ...
- eclipse的怪问题。background indexer crash recovery .java.lang.OutOfMemoryError: Java heap space
2019独角兽企业重金招聘Python工程师标准>>> 好久没用eclipse了,今天居然启动不了.一开始就出现启动不了的那个参数错误. 我就奇了怪了,虽然有个把月没用了,没动它怎么 ...
- MySQL 5.7最新版本的2个bug
好久没写博客了,都长草了.新业务上了5.7没遇到什么问题,虽然没遇到什么问题,但不代表没有问题,我有个习惯就是没事就喜欢逛逛percona的Blog,于是看到目前最新GA版本5.7.17的2个bug, ...
- HEAP: Free Heap block XXXX modified at XXXX after it was freed
开发过程中,有时候会遇到堆异常的情况 这时,VS的调试输出窗口会提示: HEAP[MemTest.exe]: HEAP: Free Heap block 39b998 modified at 39b9 ...
- PAT甲级1098 Insertion or Heap Sort：[C++题解]堆排序和插入排序
文章目录 题目分析 题目链接 题目分析 分析 插入排序的特点:前半部分有序,后半部分保持原序. 堆排序的特点: 后半部分有序,前半部分无序. 本题的bug在于,答案是唯一的,即不是插入排序就是堆排序, ...
- C++中Delete时堆错误（Heap Corruption）的原因
最近这三四天一直在跟一个bug做斗争:程序在运行过程中死掉,Output窗口显示: 1.Windows has triggered a breakpoint in ***, This may be d ...
- c++ 拷贝构造函数_禁止拷贝构造，禁止bug
禁止拷贝构造,禁止bug 一.前言 首先,我先讲讲为什么会写这篇文章:这个也是翻阅自己之前博客,当时看开源代码的时候,总是很奇怪,为什么有的代码中会会出现类似于Epoll( const Epoll&a ...
- Java里的堆(heap)栈(stack)和方法区(method)
http://imiduo.iteye.com/blog/616310 Java里的堆(heap)栈(stack)和方法区(method) <一> 基础数据类型直接在栈空间分配, 方法的 ...
- was java heap space_实战项目中Java heap space错误的解决
部标GPS通讯系统在上线之后,经过不断调试,终于稳定运行一段时间,后来又遇到了Java heap space错误异常!日志如下: 说明系统中有未释放的对象.如何找出这些未释放对象以及监控JVM堆内存, ...
最新文章
- shell脚本之case用法
- 使用坚果云同步SVN服务器数据
- python难度如何_入门Python学习难吗怎样规划学习路线
- Java开发小技巧（六）：使用Apache POI读取Excel
- rsa 模数 指数转换 c语言_模数转换，你必须知道的8个经典ADC转换电路方案
- Mac 生成SSH Key
- 401 binary watch
- kaggle房价预测特征意思_Kaggle之预测房价
- SpringBoot注解验证参数
- java队列类_用Java编写一个队列类
- matlab相对误差均值计算公式,相对误差怎么算_平均相对误差怎么计算公式
- 倒闭潮之后 快消B2B要如何做才能实现盈利
- 基于mongoose的httpclient配置SSL与cookie
- Java项目架构的演变
- R和Rstudio 下载安装
- 北斗/GPS定位模块冷启动、热启动、温启动有什么区别？
- 苹果保修期多久_手机的寿命究竟有多久，您的换机周期是否会等到手机报废呢？|手机|电池|安卓手机...
- 基于java线上教学平台计算机毕业设计源码+系统+lw文档+mysql数据库+调试部署
- 国内外深度学习开放数据集下载集合(值得收藏，不断更新)
- 162_附加145路由器项目中所用到的单链表操作_尾插、头插、遍历打印、查找ip、删除节点数据、释放全部、保存终端输入IP到文件、文件中的过滤IP插入到链表