Postgresql13.1增量排序BUG

13版本新特性新增了增量排序，本来是为了排序提升性能，但是13.1版本存在BUG
实例如下：

以上取值明显是错误的

我的测试环境13.4已经修复改BUG

大概来说，由于索引列的值出现的次数超过DEFAULT_MAX_FULL_SORT_GROUP_SIZE(64)，则会调用switchToPresortedPrefixMode函数。在这个函数中，在读取最后一个元组并判断它不属于前一个组之后，就会从for循环中中断。然而，由于lastTuple被设置为true，后续进程将错误地认为该元组已被放入 prefixsort_state

#define DEFAULT_MIN_GROUP_SIZE 32
#define DEFAULT_MAX_FULL_SORT_GROUP_SIZE (2 * DEFAULT_MIN_GROUP_SIZE)

BUG修复：
https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=82e0e29308dedbc6000e73329beb112ae7e1ad39

后面查了下release说明，已经在13.2版本修复：
https://www.postgresql.org/docs/release/13.2/

switchToPresortedPrefixMode函数：

/* ----------------------------------------------------------------* switchToPresortedPrefixMode** When we determine that we've likely encountered a large batch of tuples all* having the same presorted prefix values, we want to optimize tuplesort by* only sorting on unsorted suffix keys.** The problem is that we've already accumulated several tuples in another* tuplesort configured to sort by all columns (assuming that there may be* more than one prefix key group). So to switch to presorted prefix mode we* have to go back and look at all the tuples we've already accumulated to* verify they're all part of the same prefix key group before sorting them* solely by unsorted suffix keys.** While it's likely that all tuples already fetched are all part of a single* prefix group, we also have to handle the possibility that there is at least* one different prefix key group before the large prefix key group.* ----------------------------------------------------------------*/
static void
switchToPresortedPrefixMode(PlanState *pstate)
{IncrementalSortState *node = castNode(IncrementalSortState, pstate);ScanDirection dir;int64       nTuples = 0;bool       lastTuple = false;bool     firstTuple = true;TupleDesc    tupDesc;PlanState  *outerNode;IncrementalSort *plannode = castNode(IncrementalSort, node->ss.ps.plan);dir = node->ss.ps.state->es_direction;outerNode = outerPlanState(node);tupDesc = ExecGetResultType(outerNode);/* Configure the prefix sort state the first time around. */if (node->prefixsort_state == NULL){Tuplesortstate *prefixsort_state;int          nPresortedCols = plannode->nPresortedCols;/** Optimize the sort by assuming the prefix columns are all equal and* thus we only need to sort by any remaining columns.*/prefixsort_state = tuplesort_begin_heap(tupDesc,plannode->sort.numCols - nPresortedCols,&(plannode->sort.sortColIdx[nPresortedCols]),&(plannode->sort.sortOperators[nPresortedCols]),&(plannode->sort.collations[nPresortedCols]),&(plannode->sort.nullsFirst[nPresortedCols]),work_mem,NULL,false);node->prefixsort_state = prefixsort_state;}else{/* Next group of presorted data */tuplesort_reset(node->prefixsort_state);}/** If the current node has a bound, then it's reasonably likely that a* large prefix key group will benefit from bounded sort, so configure the* tuplesort to allow for that optimization.*/if (node->bounded){SO1_printf("Setting bound on presorted prefix tuplesort to: " INT64_FORMAT "\n",node->bound - node->bound_Done);tuplesort_set_bound(node->prefixsort_state,node->bound - node->bound_Done);}/** Copy as many tuples as we can (i.e., in the same prefix key group) from* the full sort state to the prefix sort state.*/for (;;){lastTuple = node->n_fullsort_remaining - nTuples == 1;/** When we encounter multiple prefix key groups inside the full sort* tuplesort we have to carry over the last read tuple into the next* batch.*/if (firstTuple && !TupIsNull(node->transfer_tuple)){tuplesort_puttupleslot(node->prefixsort_state, node->transfer_tuple);nTuples++;/* The carried over tuple is our new group pivot tuple. */ExecCopySlot(node->group_pivot, node->transfer_tuple);}else{tuplesort_gettupleslot(node->fullsort_state,ScanDirectionIsForward(dir),false, node->transfer_tuple, NULL);/** If this is our first time through the loop, then we need to* save the first tuple we get as our new group pivot.*/if (TupIsNull(node->group_pivot))ExecCopySlot(node->group_pivot, node->transfer_tuple);if (isCurrentGroup(node, node->group_pivot, node->transfer_tuple)){tuplesort_puttupleslot(node->prefixsort_state, node->transfer_tuple);nTuples++;}else{/** The tuple isn't part of the current batch so we need to* carry it over into the next batch of tuples we transfer out* of the full sort tuplesort into the presorted prefix* tuplesort. We don't actually have to do anything special to* save the tuple since we've already loaded it into the* node->transfer_tuple slot, and, even though that slot* points to memory inside the full sort tuplesort, we can't* reset that tuplesort anyway until we've fully transferred* out its tuples, so this reference is safe. We do need to* reset the group pivot tuple though since we've finished the* current prefix key group.*/ExecClearTuple(node->group_pivot);break;}}firstTuple = false;/** If we've copied all of the tuples from the full sort state into the* prefix sort state, then we don't actually know that we've yet found* the last tuple in that prefix key group until we check the next* tuple from the outer plan node, so we retain the current group* pivot tuple prefix key group comparison.*/if (lastTuple)break;}/** Track how many tuples remain in the full sort batch so that we know if* we need to sort multiple prefix key groups before processing tuples* remaining in the large single prefix key group we think we've* encountered.*/SO1_printf("Moving " INT64_FORMAT " tuples to presorted prefix tuplesort\n", nTuples);node->n_fullsort_remaining -= nTuples;SO1_printf("Setting n_fullsort_remaining to " INT64_FORMAT "\n", node->n_fullsort_remaining);if (lastTuple){/** We've confirmed that all tuples remaining in the full sort batch is* in the same prefix key group and moved all of those tuples into the* presorted prefix tuplesort. Now we can save our pivot comparison* tuple and continue fetching tuples from the outer execution node to* load into the presorted prefix tuplesort.*/ExecCopySlot(node->group_pivot, node->transfer_tuple);SO_printf("Setting execution_status to INCSORT_LOADPREFIXSORT (switchToPresortedPrefixMode)\n");node->execution_status = INCSORT_LOADPREFIXSORT;/** Make sure we clear the transfer tuple slot so that next time we* encounter a large prefix key group we don't incorrectly assume we* have a tuple carried over from the previous group.*/ExecClearTuple(node->transfer_tuple);}else{/** We finished a group but didn't consume all of the tuples from the* full sort state, so we'll sort this batch, let the outer node read* out all of those tuples, and then come back around to find another* batch.*/SO1_printf("Sorting presorted prefix tuplesort with " INT64_FORMAT " tuples\n", nTuples);tuplesort_performsort(node->prefixsort_state);INSTRUMENT_SORT_GROUP(node, prefixsort);if (node->bounded){/** If the current node has a bound and we've already sorted n* tuples, then the functional bound remaining is (original bound* - n), so store the current number of processed tuples for use* in configuring sorting bound.*/SO2_printf("Changing bound_Done from " INT64_FORMAT " to " INT64_FORMAT "\n",Min(node->bound, node->bound_Done + nTuples), node->bound_Done);node->bound_Done = Min(node->bound, node->bound_Done + nTuples);}SO_printf("Setting execution_status to INCSORT_READPREFIXSORT  (switchToPresortedPrefixMode)\n");node->execution_status = INCSORT_READPREFIXSORT;}
}

参考：
https://postgrespro.com/list/id/CAA3qoJ=OokS7T9K81dHSK85gKx+CbmuArf9uhrqJxT1qFjkz2Q@mail.gmail.com

Postgresql13.1增量排序BUG相关推荐

java算法----排序----（6）希尔排序（最小增量排序）
1 package log; 2 3 public class Test4 { 4 5 /** 6 * java算法---希尔排序(最小增量排序) 7 * 8 * @param args 9 */ 1 ...
谢尔排序/缩减增量排序(C++)
谢尔排序/缩减增量排序(C++) 谢尔排序/缩减增量排序: 他通过比较相距一定间隔的元素来工作,各趟比较所用的距离随着算法的进行而减小,直到只比较相邻元素的最后一趟排序为止.(好复杂) 看了一下实现代 ...
ReviewForJob——希尔排序（缩小增量排序）之塞奇威克增量序列
[0]README 0)希尔排序是基于插入排序的.将插入排序算法内for循环中的所有 1 改为增量就可以..bingo.. 插入排序源码 1)本文旨在给出希尔排序(缩小增量排序)之塞奇威克增量序列 ...
数据结构之插入排序：希尔排序(缩小增量排序)
排序算法:希尔排序.缩小增量排序思维导图: 希尔排序的定义: 例: 希尔排序d的选取: 希尔排序的代码实现: 希尔排序的性能: 思维导图: 希尔排序的定义: 普通的插入算法正序时时间复杂度会很小,但 ...
排序算法之希尔排序（缩小增量排序）
前面两篇介绍了两个非常简单又非常基础的算法--选择排序和插入排序,并通过一篇关于大乐透的小应用程序介绍了插入排序的一个简单应用.本篇介绍一个基于插入排序算法的.快速的排序算法--希尔排序.同样,本篇主 ...
Java八大排序算法之希尔排序（最小增量排序）算法
希尔排序(Shell Sort)是插入排序的一种.也称缩小增量排序,是直接插入排序算法的一种更高效的改进版本.希尔排序是非稳定排序算法.该方法因DL．Shell于1959年提出而得名. ------- ...
希尔排序（缩小增量排序）（插入排序的优化版） C++代码实现及算法分析恋上数据结构笔记
文章目录复习概要算法思想算法流程算法复杂度分析及稳定性希尔排序代码正常版希尔排序与插入排序代码对比希尔排序个人青春版(别看以免走上歧途) 复习概要算法思想与流程分这么多组分别插入排序 ...
排序算法——希尔排序（缩小增量排序）
1.希尔排序思想: 希尔排序就是把数据分成若干份子序列,从第一个元素开始,和每间隔为n的元素分成一个子序列,对每一份子序列实行直接插入排序,然后合并成一个新序列,继续对新序列以间隔m分成若干份,继续重 ...
排序算法(四)--谢尔排序(缩小增量排序)
由上一篇博文可知,冒泡排序时,若待排序序列长度为n,则冒泡排序法时间复杂度为O(n2).但若待排序序列已经按值有序,则其时间复杂度变为O(n).由此推想,若待排序序列按值"基本有序" ...

Postgresql13.1增量排序BUG

Postgresql13.1增量排序BUG相关推荐

最新文章

热门文章