Text Data for Trading—Sentiment Analysis 代码复现（二）

Test Data for Trading—Sentiment Analysis系列文章是对《Machine Learning for Algorithmic Trading》第十四章内容的讲解以及相关代码复现。因为中英文的文本分析存在较大差异，顾此系列没有选取中国市场的材料做为代码复现的数据，而是选择书后源代码进行复现。

代码复现（二）主要内容是对Textblob的各种应用的代码实现以及解释。

Textblob是一个Python库，为常见的NLP任务提供简单的API（应用程序编程接口），它是建立在自然语言工具包（nltk）和Pattern网络挖掘库的基础上的。TextBlob使语音部分标记、名词短语提取、情感分析、分类、翻译变得更加方便。

1、配置环境

import warnings
warnings.filterwarnings('ignore')#引入warnings库
#配置情绪分析所需环境
pip install textblob#引入textblob包
pip install nltk#引入nltk库
%matplotlib inline
from pathlib import Path
import numpy as np
import pandas as pd
from sklearn.feature_extraction.text import CountVectorizer
#导入sklearn用于特征提取和建模
sns.set_style('white')
np.random.seed(42)#可视化设置

为了解释TextBlob是如何使用的，我们对文章进行采样，与spaCy和其他库类似，第一步是将文件通过TextBlob管道，分配各种任务所需的注释。

path = Path('..', 'data', 'bbc')#导入所需文章
files = sorted(list(path.glob('**/*.txt')))
doc_list = []
for i, file in enumerate(files):topic = file.parts[-2]article = file.read_text(encoding='latin1').split('\n')

2、选择随机文章

article = docs.sample(1).squeeze()#抽取文章
print(f'Topic:\t{article.topic.capitalize()}\n\n{article.heading}\n')
print(article.body.strip())

抽取到的文章是： Topic: Business UK house prices dip in November UK house
prices dipped slightly in November, the Office of the Deputy Prime
Minister (ODPM) has said. The average house price fell marginally to
Â£180,226, from Â£180,444 in October. Recent evidence has suggested
that the UK housing market is slowing after interest rate increases,
and economists forecast a drop in prices during 2005. （文章节选）

文章大致讲述了英国十一月份房价下降以及经济学家做出的预测等等。

3、分词和分句处理

parsed_body = TextBlob(article.body)
parsed_body.words#抽取文章主干部分

运行该段代码可以得到一个wordlist

WordList(['UK', 'house', 'prices', 'dipped', 'slightly', 'in', 'November', 'the', 'Office', 'of', 'the', 'Deputy', 'Prime', 'Minister', 'ODPM', 'has', 'said', 'The', 'average', 'house', 'price', 'fell', 'marginally', 'to', 'Â£180,226', 'from'

parsed_body.sentences#文章句子边界检测，检测文章主体句

Sentence("UK house prices dipped slightly in November, the Office of the Deputy Prime Minister (ODPM) has said."),Sentence("The average house price fell marginally to Â£180,226, from Â£180,444 in October."),Sentence("Recent evidence has suggested that the UK housing market is slowing after interest rate increases, and economists forecast a drop in prices during 2005."),Sentence("But while the monthly figures may hint at a cooling of the market, annual house price inflation is still strong, up 13.8% in the year to November."),Sentence("Economists, however, forecast that ODPM figures are likely to show a weakening in annual house price growth in coming months."),Sentence(""Overall, the housing market activity is slowing down and that is backed up by the mortgage lending and the mortgage approvals data," said Mark Miller, at HBOS Treasury Services.")#同样的得到了一个sentence list

4、词元化处理

stemmer = SnowballStemmer('english')#初始化词元
[(word, stemmer.stem(word)) for i, word in enumerate(parsed_body.words) if word.lower() != stemmer.stem(parsed_body.words[i])]#对每一个单词进行词元化

 [('prices', 'price'),('dipped', 'dip'),('slightly', 'slight'),('November', 'novemb'),('Office', 'offic'),('Deputy', 'deputi'),('Minister', 'minist'),('average', 'averag'),('house', 'hous'),('marginally', 'margin'),('October', 'octob'),('evidence', 'evid'),('suggested', 'suggest'),('housing', 'hous'),('slowing', 'slow'),('increases', 'increas'),('economists', 'economist'),('prices', 'price')]

进行词元化处理后，得到了一张左右对照的单词表。左边是原始的单词，右侧是经过词元化处理后的单词；可以看出一些词的词元化出现了错误。比如说：increases的词元应该是increase而不是increas。

5、情感分析
Textblob进行情感分析的方式是返回一个元组 Sentiment(polarity, subjectivity). 在元组中polarity和subjective的值分别代表情绪是积极还是消极，是主观还是客观。polarity的取值范围是-1到1，-1.0代表最消极，1.0代表最积极；subjectivity的取值范围是0到1，0.0 表示最客观，1.0表示最主观。

parsed_body.sentiment#sentiment这一指标表达的是token情绪得分的平均值

Sentiment(polarity=0.10447845804988663, subjectivity=0.44258786848072557)

该文档token polarity的平均值是0.1，比较接近情绪中性：即不积极也不消极；subjectivity的平均值是0.44，也是比较中性的数值。

parsed_body.sentiment_assessments#sentiment_assessment代表的是每个划线token各自的情绪得分

Sentiment(polarity=0.10447845804988663, subjectivity=0.44258786848072557, assessments=[(['slightly'], -0.16666666666666666, 0.16666666666666666, None), (['average'], -0.15, 0.39999999999999997, None), (['recent'], 0.0, 0.25, None), (['strong'], 0.4333333333333333, 0.7333333333333333, None), (['likely'], 0.0, 1.0, None), (['overall'], 0.0, 0.0, None), (['down'], -0.15555555555555559, 0.2888888888888889, None), (['fairly'], 0.7, 0.9, None), (['nearly'], 0.1, 0.4, None), (['last'], 0.0, 0.06666666666666667, None), (['first'], 0.25, 0.3333333333333333, None), (['rose'], 0.6, 0.95, None), (['whole'], 0.2, 0.4, None), (['only'], 0.0, 1.0, None), (['second'], 0.0, 0.0, None), (['half'], -0.16666666666666666, 0.16666666666666666, None), (['overall'], 0.0, 0.0, None), (['large'], 0.21428571428571427, 0.42857142857142855, None), (['recent'], 0.0, 0.25, None), (['rose'], 0.6, 0.95, None), (['same'], 0.0, 0.125, None), (['average'], -0.15, 0.39999999999999997, None), (['more'], 0.5, 0.5, None), (['average'], -0.15, 0.39999999999999997, None), (['rose'], 0.6, 0.95, None), (['only'], 0.0, 1.0, None), (['slightly'], -0.16666666666666666, 0.16666666666666666, None), (['previous'], -0.16666666666666666, 0.16666666666666666, None)])

更多 机器学习、投资管理 相关的内容，均收录在微信公众号 HI投量化俱乐部
欢迎扫码关注

Text Data for Trading—Sentiment Analysis 代码复现（二）相关推荐

Test Data for Trading—Sentiment Analysis 代码复现（五）
Test Data for Trading-Sentiment Analysis系列文章是对<Machine Learning for Algorithmic Trading>第十四章内容 ...
细粒度情感分析（Aspect Based Sentiment Analysis, ABSA），一个从零开始的案例教程【Python实现】
目录前言数据和源码你要了解的基础 1. 细粒度情感分析(ABSA)案例背景 1.1 任务介绍 1.2 数据基本介绍 1.3 如何评估ABSA的结果 2. 任务一:Aspect Term Extr ...
[nlp] sentiment analysis(情感分析)
https://github.com/udacity/deep-learning-v2-pytorch https://github.com/udacity/deep-learning-v2-pyto ...
情感分析Sentiment Analysis 知识资料全集（入门/进阶/论文/综述/视频/专家，附查看
情感分析 ( Sentiment Analysis ) 专知荟萃入门学习进阶论文 Tutorial 综述代码视频教程领域专家入门学习斯坦福大学自然语言处理第七课"情感分析(Se ...
中文情感分析 (Sentiment Analysis) 的难点在哪？现在做得比较好的有哪几家？
点击上方,选择星标或置顶,每天给你送干货! 阅读大概需要25分钟跟随小博主,每天进步一丢丢来自: 知乎编辑: 深度学习自然语言处理公众号地址: https://www.zhihu.com/qu ...
情感分析 ( Sentiment Analysis ) 专知荟萃
入门学习斯坦福大学自然语言处理第七课"情感分析(Sentiment Analysis)" [http://52opencourse.com/235/%E6%96%AF%E5%9D ...
HAN论文模型代码复现与重构
论文简介本文主要介绍CMU在2016年发表在ACL的一篇论文:Hierarchical Attention Networks for Document Classification及其代码复现. 该 ...
Gated Mechanism for Attention Based Multi Modal Sentiment Analysis 阅读笔记
GATED MECHANISM FOR ATTENTION BASED MULTIMODAL SENTIMENT ANALYSIS 阅读笔记最近在跟进多模态的情感分析发现多模态榜一又被刷下来了,这篇 ...
【论文笔记】A comprehensive survey on sentiment analysis: Approaches, challenges and trends（中）
文章目录 3. sentiment analysis pre-processing(情感分析预处理步骤) 3.1 Data extraction(数据提取) 3.1.1 数据收集和提取 3.2 数据预 ...

Text Data for Trading—Sentiment Analysis 代码复现（二）

Text Data for Trading—Sentiment Analysis 代码复现（二）相关推荐

最新文章

热门文章