pcy 算法

PCY algorithm was developed by three Chinese scientists Park, Chen, and Yu. This is an algorithm used in the field of big data analytics for the frequent itemset mining when the dataset is very large.

PCY算法是由三位中国科学家Park，Chen和Yu开发的。 当数据集非常大时，这是在大数据分析领域中用于频繁项集挖掘的算法。

Consider we have a huge collection of data, and in this data, we have a number of transactions. For example, if we buy any product online it’s transaction is being noted. Let, a person is buying a shirt from any site now, along with the shirt the site advised the person to buy jeans also, with some discount. So, we can see that how two different things are made into a single set and associated. The main purpose of this algorithm is to make frequent item sets say, along with shirt people frequently buy jeans.

考虑一下我们有大量的数据收集，并且在这些数据中，我们有许多事务。 例如，如果我们在线购买任何产品，则会记录其交易。 现在，一个人正在从任何站点购买衬衫，该站点建议该人也购买衬衫，并有一定折扣。 因此，我们可以看到如何将两个不同的东西组合成一个单一的集合。 该算法的主要目的是使频繁的服装搭配，以及人们经常购买的衬衫。

For example:

例如：

    Transaction         Items bought
Transaction 1       Shirt + jeans
Transaction 2       Shirt + jeans +Trouser
Transaction 3       Shirt +Tie
Transaction 4       Shirt +jeans +shoes

So, from the above example we can see that shirt is most frequently bought along with jeans, so, it is considered as a frequent itemset.

因此，从上面的示例中，我们可以看到衬衫与牛仔裤一起最常购买，因此，它被认为是一件频繁的商品。

An example problem solved using PCY algorithm

使用PCY算法解决的示例问题

Question: Apply PCY algorithm on the following transaction to find the candidate sets (frequent sets).

问题：对以下事务应用PCY算法以查找候选集(频繁集)。

Given data

给定数据

    Threshold value or minimization value = 3
Hash function= (i*j) mod 10
T1  =   {1, 2, 3}
T2  =   {2, 3, 4}
T3  =   {3, 4, 5}
T4  =   {4, 5, 6}
T5  =   {1, 3, 5}
T6  =   {2, 4, 6}
T7  =   {1, 3, 4}
T8  =   {2, 4, 5}
T9  =   {3, 4, 6}
T10 =   {1, 2, 4}
T11 =   {2, 3, 5}
T12=    {3, 4, 6}

Use buckets and concepts of Mapreduce to solve the above problem.

使用存储桶和Mapreduce概念可以解决上述问题。

Solution:

解：

1. To identify the length or we can say repetition of each candidate in the given dataset.

   为了确定长度，或者我们可以说给定数据集中每个候选项的重复。

2. Reduce the candidate set to all having length 1.

   将候选集减少为所有长度为1的候选集。

3. Map pair of candidates and find the length of each pair.

   映射候选对，并找到每个对的长度。

4. Apply a hash function to find bucket no.

   应用哈希函数查找存储桶编号。

5. Draw a candidate set table.

   绘制候选集表。

Step 1: Mapping all the elements in order to find their length.

步骤1：映射所有元素以找到其长度。

    Items →    {1, 2, 3, 4, 5, 6}
Key         1  2  3  4  5  6
Value       4  6  8  8  6  4

Step 2: Removing all elements having value less than 1.

步骤2：移除所有值小于1的元素。

But here in this example there is no key having value less than 1. Hence, candidate set = {1, 2, 3, 4, 5, 6}

但是在此示例中，这里没有键的值小于1。因此，候选集= {1,2,3,4,5,6}

Step 3: Map all the candidate set in pairs and calculate their lengths.

步骤3：成对映射所有候选集并计算其长度。

    T1: {(1, 2) (1, 3) (2, 3)} = (2, 3, 3)
T2: {(2, 4) (3, 4)} = (3 4)
T3: {(3, 5) (4, 5)} = (5, 3)
T4: {(4, 5) (5, 6)} = (3, 2)
T5: {(1, 5)} = 1
T6: {(2, 6)} = 1
T7: {(1, 4)} = 2
T8: {(2, 5)} = 2
T9: {(3, 6)} = 2
T10:______
T11:______
T12:______

Note: Pairs should not get repeated avoid the pairs that are already written before.

注意：避免重复配对，避免之前已经写入过的配对。

Listing all the sets having length more than threshold value: {(1,3) (2,3) (2,4) (3,4) (3,5) (4,5) (4,6)}

列出长度大于阈值的所有集合： {(1,3)(2,3)(2,4)(3,4)(3,5)(4,5)(4,6)}

Step 4: Apply Hash Functions. (It gives us bucket number)

步骤4：应用哈希函数。 (它给了我们存储桶号)

    Hash Function = ( i * j) mod 10
(1, 3) = (1*3) mod 10 = 3
(2,3) = (2*3) mod 10 = 6
(2,4) = (2*4) mod 10 = 8
(3,4) = (3*4) mod 10 = 2
(3,5) = (3*5) mod 10 = 5
(4,5) = (4*5) mod 10 = 0
(4,6) = (4*6) mod 10 = 4

Now, arrange the pairs according to the ascending order of their obtained bucket number.

现在，根据获得的存储桶编号的升序排列对。

    Bucket no.             Pair
0                     (4,5)
2                     (3,4)
3                     (1,3)
4                     (4,6)
5                     (3,5)
6                     (2,3)
8                     (2,4)

Step 5: In this final step we will prepare the candidate set.

步骤5：在最后一步中，我们将准备候选集。

Note: Highest support count is the no. of repetition of that vector.

注意：最高支持数量为否。 该向量的重复。

Check the pairs which have the highest support count less than 3, and write those in the candidate set, if less than 3 then reject.

检查具有最高支持数的对小于3，然后将其写入候选集中，如果小于3，则拒绝。

(NOTE: There are some exceptional cases where highest count support is less than 3, i.e. threshold value and for every candidate pair write bit vector as 1 means if HCS is greater than equal to threshold then bit vector is 1 otherwise 0).

( 注：在某些特殊情况下，最高计数支持小于3，即阈值，并且对于每个候选对，将位向量写为1表示如果HCS大于等于阈值，则位向量为1，否则为0)。

Hence, The frequent itemsets are (4, 5), (3,4)

因此， 频繁项集是(4，5)，(3,4)

Conclusion:

结论：

In this article, we have discussed a very important algorithm i.e PCY algorithm used in Big Data Analytics. We have also solved a simple question to understand its application more clearly. Let me mention one more thing that this is also very important from the examination point of view, so it’s a must do the algorithm for all of you who have BDA as a subject in their academics. If you have any further queries shoot them in the comment section, will try to solve them as soon as possible. See you in my next article till then stay healthy and keep learning!

在本文中，我们讨论了非常重要的算法，即大数据分析中使用的PCY算法 。 我们还解决了一个简单的问题，以更清楚地了解其应用。 让我再说一件事，从考试的角度来看，这也是非常重要的，因此对于所有将BDA作为其学术学科的人来说，这都是必须做的算法。 如果您还有其他疑问，请在评论部分中进行介绍，请尝试尽快解决。 在下一篇文章中再见，然后保持健康并继续学习！

翻译自: https://www.includehelp.com/big-data/pcy-algorithm-in-big-data-analytics.aspx

pcy 算法