【学习笔记-时间序列预测】prophet-使用.3节日与特殊事件

1.简介

如果数据中囊括的时间段包括了假期或其他有重复性的特殊事件，必须为它们创建一个数据框(datadraft)。该数据框需要包括有两列：holiday 、ds，每出现一个节假日或特殊事件有一行。

数据框需要包括假期的所有事件，包括过去（历史数据）和未来（预测）。

2.节日影响范围设置

考虑到节日或事件的影响不一定局限在某一天，可以设置节日所影响的时间区间，在数据框中加入两列，分别是区间下界lower_window 和区间上界upper_window ，从而将假期影响值延长到范围为 [lower_window, upper_window] 的日期中。

其中，lower_window、upper_window是距离该假日的天数。

例如：

圣诞节包括平安夜，设置lower_window=-1,upper_window=0。

感恩节包括黑色星期五，设置lower_window=0,upper_window=1。

进一步地，使用prior_scale 列分别为每个假期设置先前的比例。

创建一个数据框，对'holiday','ds','lower_window','upper_window'进行定义。记录佩顿·曼宁所有季后赛出场(playoffs apperance)的日期

（数据框就是c++中的class/struct？）

playoffs = pd.DataFrame({'holiday': 'playoff','ds': pd.to_datetime(['2008-01-13', '2009-01-03', '2010-01-16','2010-01-24', '2010-02-07', '2011-01-08','2013-01-12', '2014-01-12', '2014-01-19','2014-02-02', '2015-01-11', '2016-01-17','2016-01-24', '2016-02-07']),'lower_window': 0,'upper_window': 1,
})

结果
INFO:fbprophet:Disabling daily seasonality. Run prophet with daily_seasonality=True to override this.
Initial log joint probability = -19.4685Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes99        8124.7     0.0024575       617.553      0.9029      0.9029      133Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes199       8142.36   0.000450005       130.571      0.9794      0.9794      244Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes299       8145.89   0.000143046       261.785       0.502       0.502      362Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes347       8147.56  7.64187e-005       234.984  1.791e-007       0.001      455  LS failed, Hessian reset399       8149.51    0.00013137       93.0648      0.9855      0.9855      517Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes409        8149.6  8.99008e-005       127.585  1.039e-006       0.001      573  LS failed, Hessian reset499       8151.16    0.00129732       104.076           1           1      695Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes599        8152.2     0.0165716        467.02      0.2815           1      822Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes699       8154.09  7.88016e-005       120.573           1           1      946Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes745       8154.45  5.01699e-005       165.713  1.797e-007       0.001     1062  LS failed, Hessian reset799       8154.87  8.15711e-006       72.1416           1           1     1135Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes899       8154.97  9.47403e-006       68.2667      0.4538      0.4538     1263Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes993       8155.11  1.93662e-006       71.1706  2.817e-008       0.001     1418  LS failed, Hessian reset999       8155.11  7.04455e-007       49.8225      0.6467     0.06467     1427Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes1001       8155.11  3.11142e-007       57.4737      0.1968      0.1968     1430
Optimization terminated normally:Convergence detected: relative gradient magnitude is below tolerance

将超级碗日(superbowl days) 包括在季后赛(playoff games)中。超级碗效应是附加于季后赛效应的额外影响，使用holidays 参数将假期影响包含在预测中。

其中，季后赛对当天、之后一天有影响。故upper_window设置为1，超级碗日包括在季后赛中。

superbowls = pd.DataFrame({'holiday': 'superbowl','ds': pd.to_datetime(['2010-02-07', '2014-02-02', '2016-02-07']),'lower_window': 0,'upper_window': 1,
})

3.预测

设置holidays，进行预测。

holidays = pd.concat((playoffs, superbowls))
m = Prophet(holidays=holidays)
forecast = m.fit(df).predict(future)

设置输出表格

forecast[(forecast['playoff'] + forecast['superbowl']).abs() > 0][['ds', 'playoff', 'superbowl']][-10:]

ds playoff superbowl
2190 2014-02-02 1.227170 1.203772
2191 2014-02-03 1.913055 1.460153
2532 2015-01-11 1.227170 0.000000
2533 2015-01-12 1.913055 0.000000
2901 2016-01-17 1.227170 0.000000
2902 2016-01-18 1.913055 0.000000
2908 2016-01-24 1.227170 0.000000
2909 2016-01-25 1.913055 0.000000
2922 2016-02-07 1.227170 1.203772
2923 2016-02-08 1.913055 1.460153

作图，进一步看到假期效应的影响

fig = m.plot_components(forecast)
fig.show()

可以使用 plot_forecast_component 函数（从 Python 中的 predict.plot 导入）绘制单个假期，例如 plot_forecast_component(m, forecast, 'superbowl') 以仅绘制超级碗假期组件。

4.考虑地区性节假日(Built-in Country Holidays)

指定国家/地区的名称，除了上述人为指定的假期参数外，还将包括该国家/地区的主要假期：

m = Prophet(holidays=holidays)
m.add_country_holidays(country_name='US')
m.fit(df)

结果

INFO:fbprophet:Disabling daily seasonality. Run prophet with daily_seasonality=True to override this.
Initial log joint probability = -19.4685Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes99       8150.87    0.00255627        418.81           1           1      126Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes199       8172.63     0.0049679       224.047           1           1      240Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes299          8178   0.000150487       208.217     0.03316      0.3846      354Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes375       8182.64  6.76923e-005       232.737  1.993e-007       0.001      490  LS failed, Hessian reset399       8183.31   0.000543458       137.264           1           1      518Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes499       8185.01   0.000500865       88.2412      0.8946      0.8946      649Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes541       8186.07  4.97572e-005       178.393   2.22e-007       0.001      776  LS failed, Hessian reset599       8187.19   0.000445848       175.194           1           1      844Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes648       8187.56  1.79703e-005       67.1318  1.912e-007       0.001      940  LS failed, Hessian reset682       8187.66   2.1179e-005       103.741   2.31e-007       0.001     1025  LS failed, Hessian reset699       8187.69  1.24666e-005       57.8593      0.2416      0.8861     1047Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes740        8187.7  1.13533e-005       61.4801  1.238e-007       0.001     1148  LS failed, Hessian reset780       8187.71  4.94779e-006       63.4785      0.3536           1     1197
Optimization terminated normally:Convergence detected: relative gradient magnitude is below tolerance
<fbprophet.forecaster.Prophet object at 0x000002774BB5AD60>

如果想查看Prophet模型自带的该国家的节日有哪些

m.train_holiday_names

结果

0                         playoff
1                       superbowl
2                  New Year's Day
3      Martin Luther King Jr. Day
4           Washington's Birthday
5                    Memorial Day
6                Independence Day
7                       Labor Day
8                    Columbus Day
9                    Veterans Day
10                   Thanksgiving
11                  Christmas Day
12       Christmas Day (Observed)
13        Veterans Day (Observed)
14    Independence Day (Observed)
15      New Year's Day (Observed)
dtype: object

其他国家/地区的对应名字

此外，还包括以下国家/地区的假期：

巴西 (BR)、印度尼西亚 (ID)、印度 (IN)、马来西亚 (MY)、越南 (VN)、泰国 (TH)、菲律宾 (PH)、巴基斯坦 ( PK)、孟加拉国 (BD)、埃及 (EG)、中国 (CN) 、俄罗斯 (RU)、韩国 (KR)、白俄罗斯 (BY) 和阿拉伯联合酋长国 (AE)。

加入地区性节假日设定后，训练模型并作图

forecast = m.predict(future)
fig = m.plot_components(forecast)
fig.show()

(trend的图不知道为什么跟官网给的不一样)

5.季节性的傅里叶顺序（Fourier Order for Seasonalities）

使用部分傅里叶和(a partial Fourier sum)来估计季节性。其中，部分总和的项数是决定季节性变化速度的参数。年度季节性的默认傅里叶顺序(Fourier Order)为 10，从而产生以下拟合：

from fbprophet.plot import plot_yearly
m = Prophet().fit(df)
a = plot_yearly(m)

结果

INFO:fbprophet:Disabling daily seasonality. Run prophet with daily_seasonality=True to override this.
Initial log joint probability = -19.4685Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes99       7974.84    0.00139678       451.299           1           1      127Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes199       7993.33    0.00250392       122.806           1           1      249Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes257       7996.11  6.49542e-005       196.815  3.664e-007       0.001      352  LS failed, Hessian reset299       7997.14   0.000274336       138.333      0.4632      0.4632      400Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes353        7998.5  8.92333e-005       165.006  1.585e-007       0.001      508  LS failed, Hessian reset399       7999.92   0.000379877       87.9033           1           1      568Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes499       8001.45   0.000925247       81.3781           1           1      690Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes501       8001.47  8.16398e-005       187.283  7.257e-007       0.001      737  LS failed, Hessian reset599       8003.19   0.000309044        85.014         0.2           1      865Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes629       8003.45   0.000142042       190.253  1.703e-006       0.001      943  LS failed, Hessian reset699       8003.94  3.56514e-005       75.4715           1           1     1030Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes730       8003.97  5.09062e-005       160.324  5.924e-007       0.001     1115  LS failed, Hessian reset772       8003.99   4.1733e-007       59.2228           1           1     1167
Optimization terminated normally:Convergence detected: relative gradient magnitude is below tolerance

（画不了结果图）

6.自定义季节性

此函数输入名称、季节性周期（以天为单位）以及季节性的傅里叶阶。

Prophet 中默认，傅里叶顺序为3时：表示周季节性，为10时：表示年度季节性。

m = Prophet(weekly_seasonality=False)
m.add_seasonality(name='monthly', period=30.5, fourier_order=5)
forecast = m.fit(df).predict(future)
fig = m.plot_components(forecast)
fig.show()

INFO:fbprophet:Disabling daily seasonality. Run prophet with daily_seasonality=True to override this.
Initial log joint probability = -19.4685Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes99        7784.8    0.00227015       614.046      0.2349      0.2349      128Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes199        7798.4    0.00143127       162.739           1           1      246Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes261        7799.8  3.61659e-005       104.539  2.659e-007       0.001      364  LS failed, Hessian reset299       7800.62   0.000379718       118.058           1           1      413Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes333       7801.56  6.31933e-005       176.544  2.696e-007       0.001      513  LS failed, Hessian reset391       7802.52  3.96175e-005       99.8835  5.005e-007       0.001      629  LS failed, Hessian reset399       7802.54  1.67195e-005       62.7667       1.213      0.3624      640Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes499       7804.51    0.00131752       59.3718           1           1      763Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes599        7805.4  7.98569e-005         126.1      0.5951      0.5951      884Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes636       7806.01  5.72452e-005       126.767  2.072e-007       0.001      970  LS failed, Hessian reset699       7806.67   0.000369987       149.683           1           1     1053Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes705       7806.69  4.38233e-005       107.309  3.084e-007       0.001     1106  LS failed, Hessian reset734        7806.7   4.6617e-007       66.7769      0.1981           1     1149
Optimization terminated normally:Convergence detected: relative gradient magnitude is below tolerance

7.由其他因素影响的季节性（季节性中某部分的局部季节性/例外）

对于年季节性，一年中可能有某些月份内部有特有的季节性（例如：每年夏季的数据不满足年季节性但这几个月份满足其独有的周季节性；每月的数据中，工作日跟休息日满足不同的日季节性）

即不同时间，可能还有其他因素影响数据波动，即对应的潜在规律不同。

假设淡季与旺季会影响数据趋势，在数据中加入一列描述淡季、旺季情况

def is_nfl_season(ds):date = pd.to_datetime(ds)return (date.month > 8 or date.month < 2)df['on_season'] = df['ds'].apply(is_nfl_season)
df['off_season'] = ~df['ds'].apply(is_nfl_season)

然后，屏蔽原来关于周季节性的描述，将其替换为两个将这些列指定为条件的每周季节性。

新增的描述其他影响因素的列也需要添加到待预测的未来数据帧中。

m = Prophet(weekly_seasonality=False)
m.add_seasonality(name='weekly_on_season', period=7, fourier_order=3, condition_name='on_season')
m.add_seasonality(name='weekly_off_season', period=7, fourier_order=3, condition_name='off_season')future['on_season'] = future['ds'].apply(is_nfl_season)
future['off_season'] = ~future['ds'].apply(is_nfl_season)
forecast = m.fit(df).predict(future)
fig = m.plot_components(forecast)
fig.show()

季节性仅适用于 condition_name 列为 True 的日期。

INFO:fbprophet:Disabling daily seasonality. Run prophet with daily_seasonality=True to override this.
Initial log joint probability = -19.4685Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes99       8178.83    0.00761637       397.467           1           1      131Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes199       8200.19     0.0090541       1704.06           1           1      245Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes299       8206.39     0.0115594       300.104           1           1      365Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes399       8208.97    0.00450322       109.836           1           1      477Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes411       8209.38  9.41353e-005       285.111  5.633e-007       0.001      535  LS failed, Hessian reset499       8210.88     0.0317132       548.857           1           1      637Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes536       8211.74  8.11792e-005       160.718  1.342e-007       0.001      726  LS failed, Hessian reset599       8213.11    0.00170017       240.641           1           1      802Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes699        8213.7    0.00100337       221.527      0.1273      0.1273      929Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes799       8214.69  2.70914e-005       59.7178           1           1     1061Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes863       8214.75  6.49405e-007       56.7574           1           1     1140
Optimization terminated normally:Convergence detected: relative gradient magnitude is below tolerance

从图像可以看到，在每周日打比赛的旺季，周日和周一都有大幅增加，而在淡季则完全没有。

8.假期和季节性的先前规模

如果假期项发生过拟合，调整参数holiday_prior_scale 先验比例使拟合曲线平滑。

默认情况下，此参数为 10，减少该参数能够减弱假日项的影响：

m = Prophet(holidays=holidays, holidays_prior_scale=0.05).fit(df)
forecast = m.predict(future)
forecast[(forecast['playoff'] + forecast['superbowl']).abs() > 0][['ds', 'playoff', 'superbowl']][-10:]

INFO:fbprophet:Disabling daily seasonality. Run prophet with daily_seasonality=True to override this.
Initial log joint probability = -19.4685Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes99        8116.8    0.00283116       470.914      0.7188      0.7188      132Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes199       8133.96  9.92653e-005       213.035           1           1      256Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes299       8137.94   0.000532056       213.994      0.7966      0.7966      377Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes337       8139.66    6.948e-005       167.129  1.509e-007       0.001      463  LS failed, Hessian reset399       8141.99    0.00645722       672.979           1           1      543Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes404       8142.09     0.0001227       165.145  1.817e-006       0.001      589  LS failed, Hessian reset499       8142.87     0.0028015       181.008           1           1      709Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes599       8145.14   0.000794858       177.759           1           1      843Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes680       8145.35  9.61321e-005       193.035  1.276e-006       0.001      980  LS failed, Hessian reset699       8145.38  2.62971e-006       64.4397      0.1976      0.4691     1003Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes743       8145.52   0.000119296       302.311  9.074e-007       0.001     1109  LS failed, Hessian reset797        8145.7  6.13508e-005       60.8905  7.609e-007       0.001     1219  LS failed, Hessian reset799        8145.7  1.82998e-005       63.9972      0.9115      0.9115     1221Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes854       8145.71  4.56264e-007        54.811      0.5457      0.5457     1301
Optimization terminated normally:Convergence detected: relative gradient magnitude is below tolerance

ds playoff superbowl
2190 2014-02-02 1.205772 0.962992
2191 2014-02-03 1.852125 0.992403
2532 2015-01-11 1.205772 0.000000
2533 2015-01-12 1.852125 0.000000
2901 2016-01-17 1.205772 0.000000
2902 2016-01-18 1.852125 0.000000
2908 2016-01-24 1.205772 0.000000
2909 2016-01-25 1.852125 0.000000
2922 2016-02-07 1.205772 0.962992
2923 2016-02-08 1.852125 0.992403

与之前相比，playoff/superbowl列的值明显偏小，节日效应的幅度有所降低

类似地，参数seasonality_prior_scale 也能够调整季节性模型的拟合程度。通过在假期数据框中包含一个列prior_scale，可以为各个假期单独设置先前的比例。单个季节性的先验比例可以作为参数传递给 add_seasonality。例如，可以使用以下方法设置仅每周季节性的先前比例：

m = Prophet()
m.add_seasonality(name='weekly', period=7, fourier_order=3, prior_scale=0.1)

9.额外的回归

可以使用 add_regressor 方法或函数将其他回归量添加到模型的线性部分，拟合和预测数据帧中都需要存在具有回归量值的列。

def nfl_sunday(ds):date = pd.to_datetime(ds)if date.weekday() == 6 and (date.month > 8 or date.month < 2):return 1else:return 0
df['nfl_sunday'] = df['ds'].apply(nfl_sunday)m = Prophet()
m.add_regressor('nfl_sunday')
m.fit(df)future['nfl_sunday'] = future['ds'].apply(nfl_sunday)forecast = m.predict(future)
fig = m.plot_components(forecast)