The most common method for recommendation systems often comes with Collaborating Filtering (CF) where it relies on the past user and item dataset. Two popular approaches of CF are latent factor models, which extract features from user and item matrices and neighborhood models, which finds similarities between products or users.

推荐系统最常用的方法通常是协作过滤(CF)，它依赖于过去的用户和项目数据集。 CF的两种流行方法是潜在因子模型，它们从用户和项目矩阵以及邻域模型中提取特征，从而发现产品或用户之间的相似性。

The neighborhood model is an item-oriented approach to discover the user preference based on the ratings given by the user for similar items. On the other hand, latent factor models such as Singular Value Decomposition (SVD) extract features and correlation from the user-item matrix. For example, when items are movies in different categories. SVD would generate factors when looking into the dimension space like action vs comedy, Hollywood vs Bollywood, or Marvel vs Disney. Mainly, we will focus on the latent factor model for the Singular Value Decomposition (SVD) approach.

邻域模型是一种面向项目的方法，用于根据用户对相似项目给出的评分来发现用户偏好。 另一方面，诸如奇异值分解(SVD)之类的潜在因子模型从用户项矩阵中提取特征和相关性。 例如，当项目是不同类别的电影时。 SVD在调查维度空间时会产生各种因素，例如动作与喜剧，好莱坞与宝莱坞或漫威与迪士尼。 主要，我们将专注于奇异值分解(SVD)方法的潜在因子模型。

In this article, you will learn the singular value decomposition and truncated SVD of the recommender system:

在本文中，您将学习推荐系统的奇异值分解和截断SVD：

(1) Introduction to singular value decomposition

(1)奇异值分解导论

(2) Introduction to truncated SVD

(2)截断SVD简介

(3) Hands-on experience of python code on matrix factorization

(3)关于矩阵分解的python代码的实践经验

奇异值分解导论 (Introduction to singular value decomposition)

When it comes to dimensionality reduction, the Singular Value Decomposition (SVD) is a popular method in linear algebra for matrix factorization in machine learning. Such a method shrinks the space dimension from N-dimension to K-dimension (where K<N) and reduces the number of features. SVD constructs a matrix with the row of users and columns of items and the elements are given by the users’ ratings. Singular value decomposition decomposes a matrix into three other matrices and extracts the factors from the factorization of a high-level (user-item-rating) matrix.

在降维方面，奇异值分解(SVD)是线性代数中机器学习中矩阵分解的一种流行方法。 这种方法将空间尺寸从N维缩小到K维(其中K <N)，并减少了特征数量。 SVD用用户的行和项目的列构造一个矩阵，并且元素由用户的等级给出。 奇异值分解将矩阵分解为其他三个矩阵，并从高级(用户项评级)矩阵的分解中提取因子。

Matrix U: singular matrix of (user*latent factors)Matrix S: diagonal matrix (shows the strength of each latent factor)Matrix U: singular matrix of (item*latent factors)

矩阵U：(用户*潜在因子)的奇异矩阵矩阵S：对角矩阵(显示每个潜在因子的强度)矩阵U：(项目*潜在因子)的奇异矩阵

From matrix factorization, the latent factors show the characteristics of the items. Finally, the utility matrix A is produced with shape m*n. The final output of the matrix A reduces the dimension through latent factors’ extraction. From the matrix A, it shows the relationships between users and items by mapping the user and item into r-dimensional latent space. Vector X_i is considered each item and vector Y_u is regarded as each user. The rating is given by a user on an item as R_ui = X^T_i * Y_u. The loss can be minimized by the square error difference between the product of R_ui and the expected rating.

通过矩阵分解，潜在因子显示出项目的特征。 最后，产生具有形状m * n的效用矩阵A。 矩阵A的最终输出通过潜在因子的提取减小维数。 从矩阵A，它通过将用户和项目映射到r维潜在空间来显示用户和项目之间的关系。 向量X_i被视为每个项目，向量Y_u被视为每个用户。 用户对项目给出的评级为R_ui = X ^ T_i * Y_u。 R_ui的乘积与预期额定值之间的平方误差可以使损失最小化。

Regularization is used to avoid overfitting and generalize the dataset by adding the penalty.

正则化用于避免过度拟合并通过添加惩罚来概括数据集。

Here, we add a bias term to reduce the error of actual versus predicted value by the model.

在这里，我们添加了一个偏差项，以减少模型对实际值和预测值的误差。

(u, i): user-item pairμ: the average rating of all itemsbi: average rating of item i minus μbu: the average rating given by user u minus μ

(u，i)：用户项目对μ：所有项目的平均评级bi：项目i的平均评级减去μbu：用户u给出的平均评级减去μ

The equation below adds the bias term and the regularization term:

下面的方程式将偏差项和正则项相加：

截断的SVD简介 (Introduction to truncated SVD)

When it comes to matrix factorization technique, truncated Singular Value Decomposition (SVD) is a popular method to produce features that factors a matrix M into the three matrices U, Σ, and V. Another popular method is Principal Component Analysis (PCA). Truncated SVD shares similarity with PCA while SVD is produced from the data matrix and the factorization of PCA is generated from the covariance matrix. Unlike regular SVDs, truncated SVD produces a factorization where the number of columns can be specified for a number of truncation. For example, given an n x n matrix, truncated SVD generates the matrices with the specified number of columns, whereas SVD outputs n columns of matrices.

对于矩阵分解技术，截断奇异值分解 ( SVD )是一种流行的方法，用于产生将矩阵M分解为三个矩阵U，Σ和V的特征。另一种流行的方法是主成分分析(PCA)。 截断的SVD与PCA具有相似性，而SVD是从数据矩阵生成的，而PCA的分解是从协方差矩阵生成的。 与常规SVD不同，截断的SVD会产生分解，可以为截断的数量指定列数。 例如，给定一个nxn矩阵，截短的SVD生成具有指定列数的矩阵，而SVD输出n列矩阵。

截短的SVD优于PCA的优势 (The advantages of truncated SVD over PCA)

Truncated SVD can deal with sparse matrix to generate features’ matrices, whereas PCA would operate on the entire matrix for the output of the covariance matrix.

截断的SVD可以处理稀疏矩阵以生成特征矩阵，而PCA可以对整个矩阵进行操作以输出协方差矩阵。

python代码的动手经验 (Hands-on experience of python code)

资料说明： (Data Description:)

The metadata includes 45,000 movies listed in the Full MovieLens Dataset and movies are released before July 2017. Cast, crew, plot keywords, budget, revenue, posters, release dates, languages, production companies, countries, TMDB vote counts and vote averages are in the dataset. The scale of ratings is 1–5 and obtained from the official GroupLens website. The dataset is referred to from the Kaggle dataset.

元数据包括Full MovieLens数据集中列出的45,000部电影，并且电影将于2017年7月之前发行。演员，剧组，剧情关键字，预算，收入，海报，发行日期，语言，制作公司，国家/地区，TMDB投票数和平均票数均在数据集。 评级等级为1-5，可从GroupLens官方网站获得。 该数据集是从Kaggle数据集中引用的。

使用SVD推荐电影 (Recommending movies using SVD)

Singular value decomposition (SVD) is a collaborative filtering method for movie recommendation. The aim for the code implementation is to provide users with movies’ recommendation from the latent features of item-user matrices. The code would show you how to use the SVD latent factor model for matrix factorization.

奇异值分解(SVD)是一种用于电影推荐的协作过滤方法。 代码实现的目的是根据项目用户矩阵的潜在功能为用户提供电影推荐。 该代码将向您展示如何使用SVD潜在因子模型进行矩阵分解。

数据预处理 (Data Preprocessing)

Random sample the rating dataset and generate the movie features with genres. Then, labelencode all the movies and users with respective unique ids.

随机采样评级数据集并生成具有流派的电影特征。 然后，使用各自的唯一ID对所有电影和用户进行标签编码。

num of users: 1105num of movies: 3000

模型表现 (Model Performance)

Through each run of the epoch, the rmse is reduced and the final output reaches rmse 0.57. The number of batch size would affect the number of input data fed into the model for each run. Batch size, learning rate, and regularization term are tunable to optimize the model performance.

通过每个时期，均方根值减小，最终输出达到均方根值0.57。 批处理大小的数量将影响每次运行馈入模型的输入数据的数量。 批次大小，学习率和正则项可调整以优化模型性能。

RMSE 2.1727233RMSE 2.101482RMSE 2.0310202RMSE 1.9610059RMSE 1.8911659RMSE 1.8213558RMSE 1.7515925RMSE 1.681992RMSE 1.612707RMSE 1.543902RMSE 1.4757496RMSE 1.408429RMSE 1.3421307RMSE 1.277059RMSE 1.2134355RMSE 1.1514966RMSE 1.0914934RMSE 1.0336862RMSE 0.9783424RMSE 0.9257237RMSE 0.87606686RMSE 0.82956517RMSE 0.7863303RMSE 0.7463626RMSE 0.7095342RMSE 0.67563176RMSE 0.6445249RMSE 0.6163493RMSE 0.5914116RMSE 0.5701855

使用截断的SVD推荐电影 (Recommending movies using Truncated SVD)

The first 10 components of user x movie matrix s generated through truncated SVD. There are latent features in the reconstructed matrix showing a correlation with the user ratings for the rating prediction.

通过截断的SVD生成的用户x电影矩阵s的前10个分量。 重构矩阵中存在潜在特征，这些潜在特征显示了与用户评分之间的相关性，以进行评分预测。

Since the genres column is in the list of the dictionary format, The column is preprocessed and extracted with several genres’ names separated by | format.

由于流派列在字典格式的列表中，因此该列经过预处理和提取，并使用了多个流派名称，并用|分隔。 格式。

在用户和电影矩阵上执行截断的SVD (Perform Truncated SVD on user and movie matrix)

Take a 3000 random sample of users’ ratings from the dataset and create the pivot table with the index of Userid and columns of MovieID with the rating value. Then, the user matrix is generated with 2921x1739 users by the user matrix.

从数据集中随机抽取3000个用户评分样本，并创建包含Userid索引和带有评分值的MovieID列的数据透视表。 然后，通过用户矩阵生成具有2921x1739个用户的用户矩阵。

Take 3000 random samples of movies from the dataset and create the pivot table with the index of MovieID and columns of Userid with the rating value. Then, the movie matrix is generated with 3000x1105 users by the movie matrix.

从数据集中抽取3000个电影的随机样本，并创建带有MovieID索引和带有评估值的Userid列的数据透视表。 然后，电影矩阵通过3000x1105用户生成电影矩阵。

From both user and rating matrix, 80% of data is used for training data and the rest 20% is for test data. For the train data, the reconstructed matrix is produced from 10 components of truncated SVD. The row*col length of matrices is movie_features.shape = (2400, 10) and user_features.shape = (2336, 10).

在用户矩阵和评分矩阵中，有80％的数据用于培训数据，其余20％的数据用于测试数据。 对于火车数据，从截断的SVD的10个分量生成重建的矩阵。 矩阵的行*列长度为movie_features.shape =(2400，10)和user_features.shape =(2336，10)。

TSNE Visualization

TSNE可视化

TSNE transforms the high-dimensional space of data into a low-dimensional space of data and visualizes it. Perplexity is one of the tuneable features to take the balance of local and global data and suggest the number of close neighbors each point has.

TSNE将数据的高维空间转换为数据的低维空间并对其进行可视化。 困惑是可调整的功能之一，可以平衡本地数据和全局数据并建议每个点具有的近邻数量。

Take the perplexity of 5 and 2 components of the movie features and the plot is produced and shows the clusters of movies. The correlated movies are clustered by the latent features produced from the TSNE method.

以电影特征的5个和2个部分的困惑为例，将绘制情节并显示电影的群集。 相关电影通过TSNE方法产生的潜在特征进行聚类。

准备火车和目标数据 (Prepare the train and target data)

The label of the target data is the average users’ rating and round it to 1 decimal point. There are a total of 501 movies and 1108 users’ ratings. The size of the train and the target data are data.shape = (3000, 1105) and targets.shape = (3000,).

目标数据的标签是平均用户评分，并将其舍入到小数点后一位。 共有501部电影和1108个用户评分。 火车的大小和目标数据为data.shape =(3000，1105)和targets.shape =(3000，)。

在潜在特征上训练梯度增强回归器 (Training a Gradient Boosted Regressor on latent features)

Train the model of GradientBoostingRegressor with the learning rate of 0.1 and 200 estimators. The loss function is calculated through mean squared error.

用0.1和200个估计量的学习率训练GradientBoostingRegressor模型。 损失函数通过均方误差计算。

The final prediction is the average rating of each movie by all the ratings produced from the users. The final MSE is around 0.51, which is quite optimal for the average rating model.

最终的预测是根据用户产生的所有评分，每部电影的平均评分。 最终的MSE约为0.51，对于平均评级模型而言，这是非常理想的。

       Iter       Train Loss   Remaining Time          1           0.3735            5.43s         2           0.3710            5.12s         3           0.3689            4.89s         4           0.3672            4.76s         5           0.3656            4.67s         6           0.3641            4.64s         7           0.3628            4.59s         8           0.3614            4.54s         9           0.3601            4.52s        10           0.3589            4.51s        20           0.3480            4.14s        30           0.3391            3.83s        40           0.3316            3.59s        50           0.3245            3.35s        60           0.3174            3.14s        70           0.3118            2.91s        80           0.3063            2.68s        90           0.3013            2.45s       100           0.2968            2.22s       200           0.2620            0.00sFinal MSE:0.5118555681581297

结论： (In Conclusion:)

1. Singular value decomposition decomposes three matrices and the latent factors show the characteristics of the items. It reduces the dimension through latent factors’ extraction. By adding the regularization and bias term, it optimizes the model performance by minimizing the rmse error.奇异值分解分解了三个矩阵，潜在因子显示了项的特征。 它通过潜在因素的提取来减小尺寸。 通过添加正则化和偏差项​​，它通过最小化均方根误差来优化模型性能。
2. Truncated SVD generates the matrices with the specified number of columns, whereas SVD outputs n columns of matrices. It decreases the number of output and better works on the sparse matrices for features output.截断SVD生成具有指定列数的矩阵，而SVD输出n列矩阵。 这样可以减少输出数量，并更好地处理稀疏矩阵以输出要素。