矩阵分解 SVD 和 CUR的区别
4.1SVD的定义
4.2SVD计算原理
4.2.1计算VV矩阵
对于原始数据矩阵A,
由定义可知 ΣΣ 为对角矩阵，即 Σ=ΣTΣ=ΣT ,则有
因此：
由定义可知， UU 矩阵为正交矩阵，即 UTU=IUTU=I , II 为单位矩阵，上式等价于
上式两边同时左乘 VV ,可得
而由定义可知矩阵 VV 是正交矩阵，即 VTV=IVTV=I ,I是单位矩阵，所以上式等价于：
因为矩阵 ΣΣ 是对角矩阵，因此矩阵 Σ2Σ2 也是对角矩阵，其对角元素是矩阵 ΣΣ 对应的对角元素的平方。上式表明矩阵 Σ2Σ2 的对角元素是矩阵 ATAATA 的特征值，其对应的特征向量是矩阵 VV 中的各个列向量。
4.2.2计算UU矩阵
UU矩阵的计算方法与VV矩阵的计算方法类似。
上式两边同时左乘矩阵 UU ,可得
上式表明矩阵 Σ2Σ2 的对角元素是矩阵 AATAAT 的特征值，其对应的特征向量是矩阵 UU 中的各个列向量。
A small detail needs to be explained concerning UU and VV . Each of these matrices have rr columns, while ATAATA is an n×nn×n matrix and AATAAT is an m×mm×mmatrix. Both nn and mm are at least as large as rr. Thus, AATAAT and ATAATA should have an additional n−rn−r and m−rm−r eigenpairs, respectively, and these pairs donot show up in UU, VV , and ΣΣ. Since the rank of AA is rr, all other eigenvalueswill be 0, and these are not useful.
4.3SVD的应用
Instance#1
there are two “concepts” underlying the movies: science-fiction and romance. All the boys rate only science-fiction, and all the girls rate only romance.
SVD分解如下：
The key to understanding what SVD offers is in viewing the rr columns of UU, ΣΣ, and VV as representing concepts that are hidden in the original matrix MM. In Example 11.8, these concepts are clear; one is “science fiction” and the other is “romance.” Let us think of the rows of MM as people and the columns of MM as movies. Then matrix UU connects people to concepts. For example, the person Joe, who corresponds to row 1 of MM in Fig. 11.6, likes only the concept science fiction. The value 0.14 in the first row and first column of UU is smaller than some of the other entries in that column, because while Joe watches only science fiction, he doesn’t rate those movies highly. The second column of the first row of UU is 0, because Joe doesn’t rate romance movies at all.
The matrix VV relates movies to concepts. The 0.58 in each of the first three columns of the first row of VTVT indicates that the first three movies – Matrix, Alien, and Star Wars – each are of the science-fiction genre, while the 0’s in the last two columns of the first row say that these movies do not partake of the concept romance at all. Likewise, the second row of VTVT tells us that the movies Casablanca and Titanic are exclusively romances.
Finally, the matrix ΣΣ gives the strength of each of the concepts. In our example, the strength of the science-fiction concept is 12.4, while the strength of the romance concept is 9.5. Intuitively, the science-fiction concept is stronger because the data provides more movies of that genre and more people who like them.
SVD-Interpretation #2
Dimensionality Reduction Using SVD
进行SVD分解如下：
Suppose we want to represent a very large matrix MM by its SVD components UU,ΣΣ, and VV , but these matrices are also too large to store conveniently. The best way to reduce the dimensionality of the three matrices is to set the smallest of the singular values to zero. If we set the s smallest singular values to 0, then we can also eliminate the corresponding ss rows of UU and VV .
Suppose we want to reduce the number of dimensions to two. Then we set the smallest of the singular values, which is 1.3, to zero. The effect on the expression in Fig. 11.9 is that the third column of UU and the third row of VTVT are multiplied only by 0’s when we perform the multiplication, so this row and this column may as well not be there. That is, the approximation to M'M′ obtained by using only the two largest singular values is that shown in Fig. 11.10.
The resulting matrix is quite close to the matrix M'M′ of Fig. 11.8. Ideally, the entire difference is the result of making the last singular value be 0. However, in this simple example, much of the difference is due to rounding error caused by the fact that the decomposition of M'M′ was only correct to two significant digits.
关于为什么选择较小的特征值设置为0，选择哪些个特征值并设置为0可以得到很好的降维效果，以及为什么这样处理的SVD非常有效可以参看《Mining of massive datasets》
5.PCA与SVD比较
6.CUR Decomposition(CUR分解)
In large-data applications, it is normal for the matrix MM being decomposed to be very sparse; that is, most entries are 0. For example, a matrix representing many documents (as rows) and the words they contain (as columns) will be sparse, because most words are not present in most documents. Similarly, a matrix of customers and products will be sparse because most people do not buy most products.
We cannot deal with dense matrices that have millions or billions of rows and/or columns. However, with SVD, even if MMis sparse, UU and VV will be dense. ΣΣ, being diagonal, will be sparse, but Σ is usually much smaller than UU and VV ,so its sparseness does not help.
SVD分解的劣势如下：
It is common for the matrix M that we wish to decompose to be very sparse.
But U and V from a UV or SVD decomposition will not be sparse even so.
CUR decomposition solves this problem by using only (randomly chosen) rows and columns of M.
6.1CUR分解的定义
6.1.1矩阵CC RR的构建方法
随机选择具体算法伪代码如下:
6.1.2UU矩阵的构建方法如下:
Moore-Penrose Inverse的定义如下：
CUR分解可以看成如下的优化问题：
6.2 SVD vs CUR
Ref:
https://en.wikipedia.org/wiki/CUR_matrix_approximation
http://web.stanford.edu/class/cs246/handouts.html
4.Singular-Value Decomposition（SVD）
We now take up a second form of matrix analysis that leads to a low-dimensional representation of a high-dimensional matrix. This approach, called singularvalue decomposition (SVD), allows an exact representation of any matrix, and also makes it easy to eliminate the less important parts of that representation to produce an approximate representation with any desired number of dimensions. Of course the fewer the dimensions we choose, the less accurate will be the approximation.
4.1SVD的定义
4.2SVD计算原理
4.2.1计算VV矩阵
对于原始数据矩阵A,
由定义可知 ΣΣ 为对角矩阵，即 Σ=ΣTΣ=ΣT ,则有
因此：
由定义可知， UU 矩阵为正交矩阵，即 UTU=IUTU=I , II 为单位矩阵，上式等价于
上式两边同时左乘 VV ,可得
而由定义可知矩阵 VV 是正交矩阵，即 VTV=IVTV=I ,I是单位矩阵，所以上式等价于：
因为矩阵 ΣΣ 是对角矩阵，因此矩阵 Σ2Σ2 也是对角矩阵，其对角元素是矩阵 ΣΣ 对应的对角元素的平方。上式表明矩阵 Σ2Σ2 的对角元素是矩阵 ATAATA 的特征值，其对应的特征向量是矩阵 VV 中的各个列向量。
4.2.2计算UU矩阵
UU矩阵的计算方法与VV矩阵的计算方法类似。
上式两边同时左乘矩阵 UU ,可得
上式表明矩阵 Σ2Σ2 的对角元素是矩阵 AATAAT 的特征值，其对应的特征向量是矩阵 UU 中的各个列向量。
A small detail needs to be explained concerning UU and VV . Each of these matrices have rr columns, while ATAATA is an n×nn×n matrix and AATAAT is an m×mm×mmatrix. Both nn and mm are at least as large as rr. Thus, AATAAT and ATAATA should have an additional n−rn−r and m−rm−r eigenpairs, respectively, and these pairs donot show up in UU, VV , and ΣΣ. Since the rank of AA is rr, all other eigenvalueswill be 0, and these are not useful.
4.3SVD的应用
Instance#1
there are two “concepts” underlying the movies: science-fiction and romance. All the boys rate only science-fiction, and all the girls rate only romance.
SVD分解如下：
The key to understanding what SVD offers is in viewing the rr columns of UU, ΣΣ, and VV as representing concepts that are hidden in the original matrix MM. In Example 11.8, these concepts are clear; one is “science fiction” and the other is “romance.” Let us think of the rows of MM as people and the columns of MM as movies. Then matrix UU connects people to concepts. For example, the person Joe, who corresponds to row 1 of MM in Fig. 11.6, likes only the concept science fiction. The value 0.14 in the first row and first column of UU is smaller than some of the other entries in that column, because while Joe watches only science fiction, he doesn’t rate those movies highly. The second column of the first row of UU is 0, because Joe doesn’t rate romance movies at all.
The matrix VV relates movies to concepts. The 0.58 in each of the first three columns of the first row of VTVT indicates that the first three movies – Matrix, Alien, and Star Wars – each are of the science-fiction genre, while the 0’s in the last two columns of the first row say that these movies do not partake of the concept romance at all. Likewise, the second row of VTVT tells us that the movies Casablanca and Titanic are exclusively romances.
Finally, the matrix ΣΣ gives the strength of each of the concepts. In our example, the strength of the science-fiction concept is 12.4, while the strength of the romance concept is 9.5. Intuitively, the science-fiction concept is stronger because the data provides more movies of that genre and more people who like them.
SVD-Interpretation #2
Dimensionality Reduction Using SVD
进行SVD分解如下：
Suppose we want to represent a very large matrix MM by its SVD components UU,ΣΣ, and VV , but these matrices are also too large to store conveniently. The best way to reduce the dimensionality of the three matrices is to set the smallest of the singular values to zero. If we set the s smallest singular values to 0, then we can also eliminate the corresponding ss rows of UU and VV .
Suppose we want to reduce the number of dimensions to two. Then we set the smallest of the singular values, which is 1.3, to zero. The effect on the expression in Fig. 11.9 is that the third column of UU and the third row of VTVT are multiplied only by 0’s when we perform the multiplication, so this row and this column may as well not be there. That is, the approximation to M'M′ obtained by using only the two largest singular values is that shown in Fig. 11.10.
The resulting matrix is quite close to the matrix M'M′ of Fig. 11.8. Ideally, the entire difference is the result of making the last singular value be 0. However, in this simple example, much of the difference is due to rounding error caused by the fact that the decomposition of M'M′ was only correct to two significant digits.
关于为什么选择较小的特征值设置为0，选择哪些个特征值并设置为0可以得到很好的降维效果，以及为什么这样处理的SVD非常有效可以参看《Mining of massive datasets》
5.PCA与SVD比较
6.CUR Decomposition(CUR分解)
In large-data applications, it is normal for the matrix MM being decomposed to be very sparse; that is, most entries are 0. For example, a matrix representing many documents (as rows) and the words they contain (as columns) will be sparse, because most words are not present in most documents. Similarly, a matrix of customers and products will be sparse because most people do not buy most products.
We cannot deal with dense matrices that have millions or billions of rows and/or columns. However, with SVD, even if MMis sparse, UU and VV will be dense. ΣΣ, being diagonal, will be sparse, but Σ is usually much smaller than UU and VV ,so its sparseness does not help.
SVD分解的劣势如下：
It is common for the matrix M that we wish to decompose to be very sparse.
But U and V from a UV or SVD decomposition will not be sparse even so.
CUR decomposition solves this problem by using only (randomly chosen) rows and columns of M.
6.1CUR分解的定义
6.1.1矩阵CC RR的构建方法
随机选择具体算法伪代码如下:
6.1.2UU矩阵的构建方法如下:
Moore-Penrose Inverse的定义如下：
CUR分解可以看成如下的优化问题：
6.2 SVD vs CUR
Ref:
https://en.wikipedia.org/wiki/CUR_matrix_approximation
http://web.stanford.edu/class/cs246/handouts.html
4.Singular-Value Decomposition（SVD）
We now take up a second form of matrix analysis that leads to a low-dimensional representation of a high-dimensional matrix. This approach, called singular value decomposition (SVD), allows an exact representation of any matrix, and also makes it easy to eliminate the less important parts of that representation to produce an approximate representation with any desired number of dimensions. Of course the fewer the dimensions we choose, the less accurate will be the approximation.
4.1SVD的定义
4.2SVD计算原理
4.2.1计算VV矩阵
对于原始数据矩阵A,
由定义可知 ΣΣ 为对角矩阵，即 Σ=ΣTΣ=ΣT ,则有
因此：
由定义可知， UU 矩阵为正交矩阵，即 UTU=IUTU=I , II 为单位矩阵，上式等价于
上式两边同时左乘 VV ,可得
而由定义可知矩阵 VV 是正交矩阵，即 VTV=IVTV=I ,I是单位矩阵，所以上式等价于：
因为矩阵 ΣΣ 是对角矩阵，因此矩阵 Σ2Σ2 也是对角矩阵，其对角元素是矩阵 ΣΣ 对应的对角元素的平方。上式表明矩阵 Σ2Σ2 的对角元素是矩阵 ATAATA 的特征值，其对应的特征向量是矩阵 VV 中的各个列向量。
4.2.2计算UU矩阵
UU矩阵的计算方法与VV矩阵的计算方法类似。
上式两边同时左乘矩阵 UU ,可得
上式表明矩阵 Σ2Σ2 的对角元素是矩阵 AATAAT 的特征值，其对应的特征向量是矩阵 UU 中的各个列向量。
A small detail needs to be explained concerning UU and VV . Each of these matrices have rr columns, while ATAATA is an n×nn×n matrix and AATAAT is an m×mm×mmatrix. Both nn and mm are at least as large as rr. Thus, AATAAT and ATAATA should have an additional n−rn−r and m−rm−r eigenpairs, respectively, and these pairs donot show up in UU, VV , and ΣΣ. Since the rank of AA is rr, all other eigenvalueswill be 0, and these are not useful.
4.3SVD的应用
Instance#1
there are two “concepts” underlying the movies: science-fiction and romance. All the boys rate only science-fiction, and all the girls rate only romance.
SVD分解如下：
The key to understanding what SVD offers is in viewing the rr columns of UU, ΣΣ, and VV as representing concepts that are hidden in the original matrix MM. In Example 11.8, these concepts are clear; one is “science fiction” and the other is “romance.” Let us think of the rows of MM as people and the columns of MM as movies. Then matrix UU connects people to concepts. For example, the person Joe, who corresponds to row 1 of MM in Fig. 11.6, likes only the concept science fiction. The value 0.14 in the first row and first column of UU is smaller than some of the other entries in that column, because while Joe watches only science fiction, he doesn’t rate those movies highly. The second column of the first row of UU is 0, because Joe doesn’t rate romance movies at all.
The matrix VV relates movies to concepts. The 0.58 in each of the first three columns of the first row of VTVT indicates that the first three movies – Matrix, Alien, and Star Wars – each are of the science-fiction genre, while the 0’s in the last two columns of the first row say that these movies do not partake of the concept romance at all. Likewise, the second row of VTVT tells us that the movies Casablanca and Titanic are exclusively romances.
Finally, the matrix ΣΣ gives the strength of each of the concepts. In our example, the strength of the science-fiction concept is 12.4, while the strength of the romance concept is 9.5. Intuitively, the science-fiction concept is stronger because the data provides more movies of that genre and more people who like them.
SVD-Interpretation #2
Dimensionality Reduction Using SVD
进行SVD分解如下：
Suppose we want to represent a very large matrix MM by its SVD components UU,ΣΣ, and VV , but these matrices are also too large to store conveniently. The best way to reduce the dimensionality of the three matrices is to set the smallest of the singular values to zero. If we set the s smallest singular values to 0, then we can also eliminate the corresponding ss rows of UU and VV .
Suppose we want to reduce the number of dimensions to two. Then we set the smallest of the singular values, which is 1.3, to zero. The effect on the expression in Fig. 11.9 is that the third column of UU and the third row of VTVT are multiplied only by 0’s when we perform the multiplication, so this row and this column may as well not be there. That is, the approximation to M'M′ obtained by using only the two largest singular values is that shown in Fig. 11.10.
The resulting matrix is quite close to the matrix M'M′ of Fig. 11.8. Ideally, the entire difference is the result of making the last singular value be 0. However, in this simple example, much of the difference is due to rounding error caused by the fact that the decomposition of M'M′ was only correct to two significant digits.
关于为什么选择较小的特征值设置为0，选择哪些个特征值并设置为0可以得到很好的降维效果，以及为什么这样处理的SVD非常有效可以参看《Mining of massive datasets》
5.PCA与SVD比较
6.CUR Decomposition(CUR分解)
In large-data applications, it is normal for the matrix MM being decomposed to be very sparse; that is, most entries are 0. For example, a matrix representing many documents (as rows) and the words they contain (as columns) will be sparse, because most words are not present in most documents. Similarly, a matrix of customers and products will be sparse because most people do not buy most products.
We cannot deal with dense matrices that have millions or billions of rows and/or columns. However, with SVD, even if MMis sparse, UU and VV will be dense. ΣΣ, being diagonal, will be sparse, but Σ is usually much smaller than UU and VV ,so its sparseness does not help.
SVD分解的劣势如下：
It is common for the matrix M that we wish to decompose to be very sparse.
But U and V from a UV or SVD decomposition will not be sparse even so.
CUR decomposition solves this problem by using only (randomly chosen) rows and columns of M.
6.1CUR分解的定义
6.1.1矩阵CC RR的构建方法
随机选择具体算法伪代码如下:
6.1.2UU矩阵的构建方法如下:
Moore-Penrose Inverse的定义如下：
CUR分解可以看成如下的优化问题：
6.2 SVD vs CUR
Ref:
https://en.wikipedia.org/wiki/CUR_matrix_approximation
http://web.stanford.edu/class/cs246/handouts.html
矩阵分解 SVD 和 CUR的区别相关推荐
- 【矩阵论】2. 矩阵分解——SVD
矩阵论 1. 准备知识--复数域上矩阵,Hermite变换) 1.准备知识--复数域上的内积域正交阵 1.准备知识--Hermite阵,二次型,矩阵合同,正定阵,幂0阵,幂等阵,矩阵的秩 2. 矩阵分 ...
- 【转载】推荐系统-矩阵分解-SVD－通俗易懂
[转载[https://blog.csdn.net/u011412768/article/details/52972081#commentBox] 因为要用到基于SVD的推荐作为baseline,所以 ...
- 【转】矩阵分解 - SVD
<div class="htmledit_views"> [转载[https://blog.csdn.net/u011412768/article/details/52 ...
- 矩阵分解 SVD分解
在认识SVD之前,先来学习两个相关的概念:正交矩阵和酉矩阵. 如果,则阶实矩阵称为正交矩阵.而酉矩阵是正交矩阵往复数域上的推广. 判断正交矩阵和酉矩阵的充分必要条件是:.或者说正交矩阵和酉矩阵的共轭转 ...
- 矩阵分解SVD和NMF
矩阵的秩 对于一个M×NM \times NM×N的矩阵A,其秩R(A)为线性无关的行向量(列向量)的数量.在空间中,秩表示矩阵的行向量或列向量所张成的空间的维度. 比如有矩阵并化为行最简矩阵:[12 ...
- 协同过滤与隐语义模型推荐系统实例3: 基于矩阵分解(SVD)的推荐
[ 协同过滤与隐语义模型推荐系统实例1: 数据处理 ] [ 协同过滤与隐语义模型推荐系统实例2: 基于相似度的推荐 ] 隐语义模型推荐 基于矩阵分解(SVD)的推荐 # 先计算歌曲被当前用户播放量/用 ...
- 3-19pytorch与矩阵分解SVD
- 基于SVD矩阵分解的用户商品推荐（python实现）
加粗样式## SVD矩阵分解 SVD奇异值分解 优点:简化数据,去除噪声,提高算法的结果 缺点:数据的转换可能难以理解 适用范围:数值性数据 原始数据data,我们把它分解成3个矩阵. 其中 只有对角 ...
- 矩阵分解及其Eigen实现
主要是用来记录自己的学习过程,内容也主要来自于网上的各种资料,然后自己总结而来,参考的资料都以注明,感谢这些作者的分享.如果内容有误,请大家指点. LU分解1 理论 定义 将矩阵等价为两 ...
最新文章
- SQL,NoSQL优缺点总结
- [问题解决]NotImplementedError 错误原因：子类没有实现父类要求一定要实现的接口
- NOI前总结：点分治
- NYOJ 298 点的变换（矩阵快速幂）
- 基于linux智能家居系统设计,基于Linux的智能家居的设计(2)
- 使用PHP CURL 模拟HTTP实现在线请求工具-toolfk程序员工具网
- python原理与架构_Python：爬虫原理和网页构造
- （47）Xilinx VIO IP核配置（八）（第10天）
- QT每日一练day29：QT中的多线程探究
- javascript获取窗口位置、绝对位置、事件位置等
- VC知识库人物专访：搜狗CEO兼任搜狐CTO王小川
- 常用正交表（可直接复制）以及混合正交表的使用
- 贪心算法 --- 例题2.哈夫曼编码问题
- 202. 快乐数 (Python 实现)
- 揭秘中国球员十大豪宅
- 华大单片机HC32L130J6TA入坑全纪录（一）
- 618买什么蓝牙耳机最划算？四款高品质蓝牙耳机测评
- [含lw+源码等]javaweb银行柜员业务绩效考核系统
- [转]程序员收集整理的PHP资源大全，包含各种类库及框架等
- 开源虚拟机Bochs安装以及踩坑