使用CNN进行文本分类

卷积神经网络

英文邮件分类

语料

simplistic , silly and tedious .
it's so laddish and juvenile , only teenage boys could possibly find it funny .
exploitative and largely devoid of the depth or sophistication that would make watching such a graphic treatment of the crimes bearable .
[garbus] discards the potential for pathological study , exhuming instead , the skewed melodrama of the circumstantial situation .
a visually flashy but narratively opaque and emotionally vapid exercise in style and mystification .
the story is also as unoriginal as they come , already having been recycled more times than i'd care to count .
about the only thing to give the movie points for is bravado -- to take an entirely stale concept and push it through the audience's meat grinder one more time .
not so much farcical as sour .
unfortunately the story and the actors are served with a hack script .
all the more disquieting for its relatively gore-free allusions to the serial murders , but it falls down in its attempts to humanize its subject .
a sentimental mess that never rings true .
while the performances are often engaging , this loose collection of largely improvised numbers would probably have worked better as a one-hour tv documentary .
interesting , but not compelling .
on a cutting room floor somewhere lies . . . footage that might have made no such thing a trenchant , ironic cultural satire instead of a frustrating misfire .
while the ensemble player who gained notice in guy ritchie's lock , stock and two smoking barrels and snatch has the bod , he's unlikely to become a household name on the basis of his first starring vehicle .
there is a difference between movies with the courage to go over the top and movies that don't care about being stupid
nothing here seems as funny as it did in analyze this , not even joe viterelli as de niro's right-hand goombah .
such master screenwriting comes courtesy of john pogue , the yale grad who previously gave us " the skulls " and last year's " rollerball . " enough said , except : film overboard !
here , common sense flies out the window , along with the hail of bullets , none of which ever seem to hit sascha .
this 100-minute movie only has about 25 minutes of decent material .
the execution is so pedestrian that the most positive comment we can make is that rob schneider actually turns in a pretty convincing performance as a prissy teenage girl .
on its own , it's not very interesting . as a remake , it's a pale imitation .
it shows that some studios firmly believe that people have lost the ability to think and will forgive any shoddy product as long as there's a little girl-on-girl action .
a farce of a parody of a comedy of a premise , it isn't a comparison to reality so much as it is a commentary about our knowledge of films .
as exciting as all this exoticism might sound to the typical pax viewer , the rest of us will be lulled into a coma .
the party scenes deliver some tawdry kicks . the rest of the film . . . is dudsville .
our culture is headed down the toilet with the ferocity of a frozen burrito after an all-night tequila bender — and i know this because i've seen 'jackass : the movie . '
the criticism never rises above easy , cynical potshots at morally bankrupt characters . . .
the movie's something-borrowed construction feels less the product of loving , well integrated homage and more like a mere excuse for the wan , thinly sketched story . killing time , that's all that's going on here .
stupid , infantile , redundant , sloppy , over-the-top , and amateurish . yep , it's " waking up in reno . " go back to sleep .
somewhere in the middle , the film compels , as demme experiments he harvests a few movie moment gems , but the field of roughage dominates .
the action clichés just pile up .
、、、、、

data_helpers.py

import numpy as np
import re
import itertools
from collections import Counterdef clean_str(string):"""Tokenization/string cleaning for all datasets except for SST.Original taken from https://github.com/yoonkim/CNN_sentence/blob/master/process_data.py"""string = re.sub(r"[^A-Za-z0-9(),!?\'\`]", " ", string)string = re.sub(r"\'s", " \'s", string)string = re.sub(r"\'ve", " \'ve", string)string = re.sub(r"n\'t", " n\'t", string)string = re.sub(r"\'re", " \'re", string)string = re.sub(r"\'d", " \'d", string)string = re.sub(r"\'ll", " \'ll", string)string = re.sub(r",", " , ", string)string = re.sub(r"!", " ! ", string)string = re.sub(r"\(", " \( ", string)string = re.sub(r"\)", " \) ", string)string = re.sub(r"\?", " \? ", string)string = re.sub(r"\s{2,}", " ", string)return string.strip().lower()def load_data_and_labels(positive_data_file, negative_data_file):"""Loads MR polarity data from files, splits the data into words and generates labels.Returns split sentences and labels."""# Load data from filespositive = open(positive_data_file, "rb").read().decode('utf-8')negative = open(negative_data_file, "rb").read().decode('utf-8')positive_examples = positive.split('\n')[:-1]negative_examples = negative.split('\n')[:-1]positive_examples = [s.strip() for s in positive_examples]negative_examples = [s.strip() for s in negative_examples]#positive_examples = list(open(positive_data_file, "rb").read().decode('utf-8'))#positive_examples = [s.strip() for s in positive_examples]#negative_examples = list(open(negative_data_file, "rb").read().decode('utf-8'))#negative_examples = [s.strip() for s in negative_examples]# Split by wordsx_text = positive_examples + negative_examplesx_text = [clean_str(sent) for sent in x_text]# Generate labelspositive_labels = [[0, 1] for _ in positive_examples]negative_labels = [[1, 0] for _ in negative_examples]y = np.concatenate([positive_labels, negative_labels], 0)return [x_text, y]def batch_iter(data, batch_size, num_epochs, shuffle=True):"""Generates a batch iterator for a dataset."""data = np.array(data)data_size = len(data)num_batches_per_epoch = int((len(data)-1)/batch_size) + 1for epoch in range(num_epochs):# Shuffle the data at each epochif shuffle:shuffle_indices = np.random.permutation(np.arange(data_size))shuffled_data = data[shuffle_indices]else:shuffled_data = datafor batch_num in range(num_batches_per_epoch):start_index = batch_num * batch_sizeend_index = min((batch_num + 1) * batch_size, data_size)yield shuffled_data[start_index:end_index]

text_cnn.py

import tensorflow as tf
import numpy as npclass TextCNN(object):"""A CNN for text classification.Uses an embedding layer, followed by a convolutional, max-pooling and softmax layer."""def __init__(self, sequence_length, num_classes, vocab_size,embedding_size, filter_sizes, num_filters, l2_reg_lambda=0.0):# Placeholders for input, output and dropoutself.input_x = tf.placeholder(tf.int32, [None, sequence_length], name="input_x")self.input_y = tf.placeholder(tf.float32, [None, num_classes], name="input_y")self.dropout_keep_prob = tf.placeholder(tf.float32, name="dropout_keep_prob")# Keeping track of l2 regularization loss (optional)l2_loss = tf.constant(0.0)# Embedding layerwith tf.device('/cpu:0'), tf.name_scope("embedding"):self.W = tf.Variable(tf.random_uniform([vocab_size, embedding_size], -1.0, 1.0),name="W")self.embedded_chars = tf.nn.embedding_lookup(self.W, self.input_x)self.embedded_chars_expanded = tf.expand_dims(self.embedded_chars, -1)# Create a convolution + maxpool layer for each filter sizepooled_outputs = []for i, filter_size in enumerate(filter_sizes):with tf.name_scope("conv-maxpool-%s" % filter_size):# Convolution Layerfilter_shape = [filter_size, embedding_size, 1, num_filters]W = tf.Variable(tf.truncated_normal(filter_shape, stddev=0.1), name="W")b = tf.Variable(tf.constant(0.1, shape=[num_filters]), name="b")conv = tf.nn.conv2d(self.embedded_chars_expanded,W,strides=[1, 1, 1, 1],padding="VALID",name="conv")# Apply nonlinearityh = tf.nn.relu(tf.nn.bias_add(conv, b), name="relu")# Maxpooling over the outputspooled = tf.nn.max_pool(h,ksize=[1, sequence_length - filter_size + 1, 1, 1],strides=[1, 1, 1, 1],padding='VALID',name="pool")pooled_outputs.append(pooled)# Combine all the pooled featuresnum_filters_total = num_filters * len(filter_sizes)#self.h_pool = tf.concat(pooled_outputs, 3)self.h_pool = tf.concat(3, pooled_outputs)self.h_pool_flat = tf.reshape(self.h_pool, [-1, num_filters_total])# Add dropoutwith tf.name_scope("dropout"):self.h_drop = tf.nn.dropout(self.h_pool_flat, self.dropout_keep_prob)# Final (unnormalized) scores and predictionswith tf.name_scope("output"):W = tf.get_variable("W",shape=[num_filters_total, num_classes],initializer=tf.contrib.layers.xavier_initializer())b = tf.Variable(tf.constant(0.1, shape=[num_classes]), name="b")l2_loss += tf.nn.l2_loss(W)l2_loss += tf.nn.l2_loss(b)self.scores = tf.nn.xw_plus_b(self.h_drop, W, b, name="scores")self.predictions = tf.argmax(self.scores, 1, name="predictions")# CalculateMean cross-entropy losswith tf.name_scope("loss"):losses = tf.nn.softmax_cross_entropy_with_logits(logits=self.scores, labels=self.input_y)self.loss = tf.reduce_mean(losses) + l2_reg_lambda * l2_loss# Accuracywith tf.name_scope("accuracy"):correct_predictions = tf.equal(self.predictions, tf.argmax(self.input_y, 1))self.accuracy = tf.reduce_mean(tf.cast(correct_predictions, "float"), name="accuracy")

train.py

#! /usr/bin/env pythonimport tensorflow as tf
import numpy as np
import os
import time
import datetime
import data_helpers
from text_cnn import TextCNN
from tensorflow.contrib import learn# Parameters
# ==================================================# Data loading params
tf.flags.DEFINE_float("dev_sample_percentage", .1, "Percentage of the training data to use for validation")
tf.flags.DEFINE_string("positive_data_file", "./data/rt-polaritydata/rt-polarity.pos", "Data source for the positive data.")
tf.flags.DEFINE_string("negative_data_file", "./data/rt-polaritydata/rt-polarity.neg", "Data source for the negative data.")# Model Hyperparameters
tf.flags.DEFINE_integer("embedding_dim", 128, "Dimensionality of character embedding (default: 128)")
tf.flags.DEFINE_string("filter_sizes", "3,4,5", "Comma-separated filter sizes (default: '3,4,5')")
tf.flags.DEFINE_integer("num_filters", 128, "Number of filters per filter size (default: 128)")
tf.flags.DEFINE_float("dropout_keep_prob", 0.5, "Dropout keep probability (default: 0.5)")
tf.flags.DEFINE_float("l2_reg_lambda", 0.0, "L2 regularization lambda (default: 0.0)")# Training parameters
tf.flags.DEFINE_integer("batch_size", 64, "Batch Size (default: 64)")
tf.flags.DEFINE_integer("num_epochs", 200, "Number of training epochs (default: 200)")
tf.flags.DEFINE_integer("evaluate_every", 100, "Evaluate model on dev set after this many steps (default: 100)")
tf.flags.DEFINE_integer("checkpoint_every", 100, "Save model after this many steps (default: 100)")
tf.flags.DEFINE_integer("num_checkpoints", 5, "Number of checkpoints to store (default: 5)")
# Misc Parameters
tf.flags.DEFINE_boolean("allow_soft_placement", True, "Allow device soft device placement")
tf.flags.DEFINE_boolean("log_device_placement", False, "Log placement of ops on devices")FLAGS = tf.flags.FLAGS
FLAGS._parse_flags()
print("\nParameters:")
for attr, value in sorted(FLAGS.__flags.items()):print("{}={}".format(attr.upper(), value))
print("")# Data Preparation
# ==================================================# Load data
print("Loading data...")
x_text, y = data_helpers.load_data_and_labels(FLAGS.positive_data_file, FLAGS.negative_data_file)# Build vocabulary
max_document_length = max([len(x.split(" ")) for x in x_text])
vocab_processor = learn.preprocessing.VocabularyProcessor(max_document_length)
x = np.array(list(vocab_processor.fit_transform(x_text)))# Randomly shuffle data
np.random.seed(10)
shuffle_indices = np.random.permutation(np.arange(len(y)))
x_shuffled = x[shuffle_indices]
y_shuffled = y[shuffle_indices]# Split train/test set
# TODO: This is very crude, should use cross-validation
dev_sample_index = -1 * int(FLAGS.dev_sample_percentage * float(len(y)))
x_train, x_dev = x_shuffled[:dev_sample_index], x_shuffled[dev_sample_index:]
y_train, y_dev = y_shuffled[:dev_sample_index], y_shuffled[dev_sample_index:]
print("Vocabulary Size: {:d}".format(len(vocab_processor.vocabulary_)))
print("Train/Dev split: {:d}/{:d}".format(len(y_train), len(y_dev)))# Training
# ==================================================with tf.Graph().as_default():session_conf = tf.ConfigProto(allow_soft_placement=FLAGS.allow_soft_placement,log_device_placement=FLAGS.log_device_placement)sess = tf.Session(config=session_conf)with sess.as_default():cnn = TextCNN(sequence_length=x_train.shape[1],num_classes=y_train.shape[1],vocab_size=len(vocab_processor.vocabulary_),embedding_size=FLAGS.embedding_dim,filter_sizes=list(map(int, FLAGS.filter_sizes.split(","))),num_filters=FLAGS.num_filters,l2_reg_lambda=FLAGS.l2_reg_lambda)# Define Training procedureglobal_step = tf.Variable(0, name="global_step", trainable=False)optimizer = tf.train.AdamOptimizer(1e-3)grads_and_vars = optimizer.compute_gradients(cnn.loss)train_op = optimizer.apply_gradients(grads_and_vars, global_step=global_step)# Keep track of gradient values and sparsity (optional)grad_summaries = []for g, v in grads_and_vars:if g is not None:grad_hist_summary = tf.summary.histogram("{}/grad/hist".format(v.name), g)sparsity_summary = tf.summary.scalar("{}/grad/sparsity".format(v.name), tf.nn.zero_fraction(g))grad_summaries.append(grad_hist_summary)grad_summaries.append(sparsity_summary)grad_summaries_merged = tf.summary.merge(grad_summaries)# Output directory for models and summariestimestamp = str(int(time.time()))out_dir = os.path.abspath(os.path.join(os.path.curdir, "runs", timestamp))print("Writing to {}\n".format(out_dir))# Summaries for loss and accuracyloss_summary = tf.summary.scalar("loss", cnn.loss)acc_summary = tf.summary.scalar("accuracy", cnn.accuracy)# Train Summariestrain_summary_op = tf.summary.merge([loss_summary, acc_summary, grad_summaries_merged])train_summary_dir = os.path.join(out_dir, "summaries", "train")train_summary_writer = tf.summary.FileWriter(train_summary_dir, sess.graph)# Dev summariesdev_summary_op = tf.summary.merge([loss_summary, acc_summary])dev_summary_dir = os.path.join(out_dir, "summaries", "dev")dev_summary_writer = tf.summary.FileWriter(dev_summary_dir, sess.graph)# Checkpoint directory. Tensorflow assumes this directory already exists so we need to create itcheckpoint_dir = os.path.abspath(os.path.join(out_dir, "checkpoints"))checkpoint_prefix = os.path.join(checkpoint_dir, "model")if not os.path.exists(checkpoint_dir):os.makedirs(checkpoint_dir)saver = tf.train.Saver(tf.global_variables(), max_to_keep=FLAGS.num_checkpoints)# Write vocabularyvocab_processor.save(os.path.join(out_dir, "vocab"))# Initialize all variablessess.run(tf.global_variables_initializer())def train_step(x_batch, y_batch):"""A single training step"""feed_dict = {cnn.input_x: x_batch,cnn.input_y: y_batch,cnn.dropout_keep_prob: FLAGS.dropout_keep_prob}_, step, summaries, loss, accuracy = sess.run([train_op, global_step, train_summary_op, cnn.loss, cnn.accuracy],feed_dict)time_str = datetime.datetime.now().isoformat()print("{}: step {}, loss {:g}, acc {:g}".format(time_str, step, loss, accuracy))train_summary_writer.add_summary(summaries, step)def dev_step(x_batch, y_batch, writer=None):"""Evaluates model on a dev set"""feed_dict = {cnn.input_x: x_batch,cnn.input_y: y_batch,cnn.dropout_keep_prob: 1.0}step, summaries, loss, accuracy = sess.run([global_step, dev_summary_op, cnn.loss, cnn.accuracy],feed_dict)time_str = datetime.datetime.now().isoformat()print("{}: step {}, loss {:g}, acc {:g}".format(time_str, step, loss, accuracy))if writer:writer.add_summary(summaries, step)# Generate batchesbatches = data_helpers.batch_iter(list(zip(x_train, y_train)), FLAGS.batch_size, FLAGS.num_epochs)# Training loop. For each batch...for batch in batches:x_batch, y_batch = zip(*batch)train_step(x_batch, y_batch)current_step = tf.train.global_step(sess, global_step)if current_step % FLAGS.evaluate_every == 0:print("\nEvaluation:")dev_step(x_dev, y_dev, writer=dev_summary_writer)print("")if current_step % FLAGS.checkpoint_every == 0:path = saver.save(sess, './', global_step=current_step)print("Saved model checkpoint to {}\n".format(path))

eval.py

#! /usr/bin/env pythonimport tensorflow as tf
import numpy as np
import os
import time
import datetime
import data_helpers
from text_cnn import TextCNN
from tensorflow.contrib import learn
import csv# Parameters
# ==================================================# Data Parameters
tf.flags.DEFINE_string("positive_data_file", "./data/rt-polaritydata/rt-polarity.pos", "Data source for the positive data.")
tf.flags.DEFINE_string("negative_data_file", "./data/rt-polaritydata/rt-polarity.neg", "Data source for the positive data.")# Eval Parameters
tf.flags.DEFINE_integer("batch_size", 64, "Batch Size (default: 64)")
tf.flags.DEFINE_string("checkpoint_dir", "./", "Checkpoint directory from training run")
tf.flags.DEFINE_boolean("eval_train", False, "Evaluate on all training data")# Misc Parameters
tf.flags.DEFINE_boolean("allow_soft_placement", True, "Allow device soft device placement")
tf.flags.DEFINE_boolean("log_device_placement", False, "Log placement of ops on devices")FLAGS = tf.flags.FLAGS
FLAGS._parse_flags()
print("\nParameters:")
for attr, value in sorted(FLAGS.__flags.items()):print("{}={}".format(attr.upper(), value))
print("")# CHANGE THIS: Load data. Load your own data here
if FLAGS.eval_train:x_raw, y_test = data_helpers.load_data_and_labels(FLAGS.positive_data_file, FLAGS.negative_data_file)y_test = np.argmax(y_test, axis=1)
else:x_raw = ["a masterpiece four years in the making", "everything is off."]y_test = [1, 0]# Map data into vocabulary
vocab_path = os.path.join(FLAGS.checkpoint_dir, "..", "vocab")
vocab_processor = learn.preprocessing.VocabularyProcessor.restore(vocab_path)
x_test = np.array(list(vocab_processor.transform(x_raw)))print("\nEvaluating...\n")# Evaluation
# ==================================================
checkpoint_file = tf.train.latest_checkpoint(FLAGS.checkpoint_dir)
graph = tf.Graph()
with graph.as_default():session_conf = tf.ConfigProto(allow_soft_placement=FLAGS.allow_soft_placement,log_device_placement=FLAGS.log_device_placement)sess = tf.Session(config=session_conf)with sess.as_default():# Load the saved meta graph and restore variablessaver = tf.train.import_meta_graph("{}.meta".format(checkpoint_file))saver.restore(sess, checkpoint_file)# Get the placeholders from the graph by nameinput_x = graph.get_operation_by_name("input_x").outputs[0]# input_y = graph.get_operation_by_name("input_y").outputs[0]dropout_keep_prob = graph.get_operation_by_name("dropout_keep_prob").outputs[0]# Tensors we want to evaluatepredictions = graph.get_operation_by_name("output/predictions").outputs[0]# Generate batches for one epochbatches = data_helpers.batch_iter(list(x_test), FLAGS.batch_size, 1, shuffle=False)# Collect the predictions hereall_predictions = []for x_test_batch in batches:batch_predictions = sess.run(predictions, {input_x: x_test_batch, dropout_keep_prob: 1.0})all_predictions = np.concatenate([all_predictions, batch_predictions])# Print accuracy if y_test is defined
if y_test is not None:correct_predictions = float(sum(all_predictions == y_test))print("Total number of test examples: {}".format(len(y_test)))print("Accuracy: {:g}".format(correct_predictions/float(len(y_test))))# Save the evaluation to a csv
predictions_human_readable = np.column_stack((np.array(x_raw), all_predictions))
out_path = os.path.join(FLAGS.checkpoint_dir, "..", "prediction.csv")
print("Saving evaluation to {0}".format(out_path))
with open(out_path, 'w') as f:csv.writer(f).writerows(predictions_human_readable)

中文邮件分类

语料

本公司有部分普通发票（商品销售发票）增值税发票及海关代征增值税专用缴款书及其它服务行业发票, 公路、内河运输发票。可以以低税率为贵公司代开，本公司具有内、外贸生意实力，保证我司开具的票据的真实性。 希望可以合作!共同发展!敬侯您的来电洽谈、咨询！ 联系人：李先生      联系电话：13632588281 如有打扰望谅解，祝商琪。
本公司有部分普通发票（商品销售发票）增值税发票及海关代征增值税专用缴款书及其它服务行业发票, 公路、内河运输发票。可以以低税率为贵公司代开，本公司具有内、外贸生意实力，保证我司开具的票据的真实性。 希望可以合作!共同发展!敬侯您的来电洽谈、咨询！ 联系人：李先生      联系电话：13632588281 如有打扰望谅解，祝商琪。
网址：www.8wx8.com,电话：0551-3961292，QQ：415455699 2、网站制作（复制，你所见的网站，就是你的网上家园） 为您提供最可靠的域名,主机服务，网站建设,邮件群发，百度首页推广，发广告服务等…… 网址：www.hf263.net,电话：0551-3961292，QQ：415455699 ########价格全国最低#################### 客户就是上帝，我希望所有我接待的客户都成为我的朋友！
本公司有部分普通发票（商品销售发票）增值税发票及海关代征增值税专用缴款书及其它服务行业发票, 公路、内河运输发票。可以以低税率为贵公司代开，本公司具有内、外贸生意实力，保证我司开具的票据的真实性。 希望可以合作!共同发展!敬侯您的来电洽谈、咨询！ 联系人：李先生      联系电话：13632588281 如有打扰望谅解，祝商琪。
本公司有部分普通发票（商品销售发票）增值税发票及海关代征增值税专用缴款书及其它服务行业发票, 公路、内河运输发票。可以以低税率为贵公司代开，本公司具有内、外贸生意实力，保证我司开具的票据的真实性。 希望可以合作!共同发展!敬侯您的来电洽谈、咨询！ 联系人： 李先生      联系电话：13632588281 如有打扰望谅解，祝商琪。
贵公司经理 你好！ 本公司为广州市荣兴贸易有限公司，本着双方公司利益的出发点可向外代开 普通国（地）税发票，商品销售发票，运输发票，餐饮发票，广告发票，服务行业， 等等．．．．．．普通发票只收２％个税点． 如贵公司在这方面上有需要可来电咨询，欢迎诚意的合作． 联系人：唐娜 联系电话：020-33520954 联系手机：13533999436
本公司有部分普通发票（商品销售发票）增值税发票及海关代征增值税专用缴款书及其它服务行业发票, 公路、内河运输发票。可以以低税率为贵公司代开，本公司具有内、外贸生意实力，保证我司开具的票据的真实性。 希望可以合作!共同发展!敬侯您的来电洽谈、咨询！ 联系人：李先生      联系电话：13632588281 如有打扰望谅解，祝商琪。
本公司有部分普通发票（商品销售发票）增值税发票及海关代征增值税专用缴款书及其它服务行业发票, 公路、内河运输发票。可以以低税率为贵公司代开，本公司具有内、外贸生意实力，保证我司开具的票据的真实性。 希望可以合作!共同发展!敬侯您的来电洽谈、咨询！ 联系人：李先生      联系电话：13632588281 如有打扰望谅解，祝商琪。
您好， 邮件无法发送到您指定的地址：andrew@vip.sina.com 该邮箱内存储信件过多已超出所限制的使用空间，无法再接收新邮件。 请您通知该邮箱用户及时清理自己邮箱内的信件，以避免因接收不到新邮件而带来的不必要的损失。谢谢您的配合。 --- 附件中的内容是原信件的一份拷贝 The message contains Unicode characters and has been sent as a binary attachment.
本公司有部分普通发票（商品销售发票）增值税发票及海关代征增值税专用缴款书及其它服务行业发票, 公路、内河运输发票。可以以低税率为贵公司代开，本公司具有内、外贸生意实力，保证我司开具的票据的真实性。 希望可以合作!共同发展!敬侯您的来电洽谈、咨询！ 联系人： 李先生      联系电话：13632588281 如有打扰望谅解，祝商琪。
妇科囊肿不开刀,盆腔积液几瓶消. 新乡市花园妇幼门诊多年来研制的中 药对治疗妇科囊肿,盆腔炎,附件炎,输 卵管炎,输卵管阻塞不孕有特效. 本门诊开展多种妇科手术,技高,诚信, 优质,安全,无痛人流术更是技高一筹, {就象睡了几分钟}"妇炎二号"名声在外, 不少人千里买药,真是"谁用谁知道"..... 祥情请点击www.hyfymz.com 中文网址:花园妇幼门诊
妇科囊肿不开刀,盆腔积液几瓶消. 新乡市花园妇幼门诊多年来研制的中 药对治疗妇科囊肿,盆腔炎,附件炎,输 卵管炎,输卵管阻塞不孕有特效. 本门诊开展多种妇科手术,技高,诚信, 优质,安全,无痛人流术更是技高一筹, {就象睡了几分钟}"妇炎二号"名声在外, 不少人千里买药,真是"谁用谁知道"..... 祥情请点击www.hyfymz.com 中文网址:花园妇幼门诊
妇科囊肿不开刀,盆腔积液几瓶消. 新乡市花园妇幼门诊多年来研制的中 药对治疗妇科囊肿,盆腔炎,附件炎,输 卵管炎,输卵管阻塞不孕有特效. 本门诊开展多种妇科手术,技高,诚信, 优质,安全,无痛人流术更是技高一筹, {就象睡了几分钟}"妇炎二号"名声在外, 不少人千里买药,真是"谁用谁知道"..... 祥情请点击www.hyfymz.com 中文网址:花园妇幼门诊
本公司有部分普通发票（商品销售发票）增值税发票及海关代征增值税专用缴款书及其它服务行业发票, 公路、内河运输发票。可以以低税率为贵公司代开，本公司具有内、外贸生意实力，保证我司开具的票据的真实性。 希望可以合作!共同发展!敬侯您的来电洽谈、咨询！ 联系人：李先生      联系电话：13632588281 如有打扰望谅解，祝商琪。
妇科囊肿不开刀,盆腔积液几瓶消. 新乡市花园妇幼门诊多年来研制的中 药对治疗妇科囊肿,盆腔炎,附件炎,输 卵管炎,输卵管阻塞不孕有特效. 本门诊开展多种妇科手术,技高,诚信, 优质,安全,无痛人流术更是技高一筹, {就象睡了几分钟}"妇炎二号"名声在外, 不少人千里买药,真是"谁用谁知道"..... 祥情请点击www.hyfymz.com 中文网址:花园妇幼门诊
妇科囊肿不开刀,盆腔积液几瓶消. 新乡市花园妇幼门诊多年来研制的中 药对治疗妇科囊肿,盆腔炎,附件炎,输 卵管炎,输卵管阻塞不孕有特效. 本门诊开展多种妇科手术,技高,诚信, 优质,安全,无痛人流术更是技高一筹, {就象睡了几分钟}"妇炎二号"名声在外, 不少人千里买药,真是"谁用谁知道"..... 祥情请点击www.hyfymz.com 中文网址:花园妇幼门诊
本公司有部分普通发票（商品销售发票）增值税发票及海关代征增值税专用缴款书及其它服务行业发票, 公路、内河运输发票。可以以低税率为贵公司代开，本公司具有内、外贸生意实力，保证我司开具的票据的真实性。 希望可以合作!共同发展!敬侯您的来电洽谈、咨询！ 联系人： 李先生      联系电话：13632588281 如有打扰望谅解，祝商琪。
贵公司财务(经理); 本公司是<<深圳市海天实业有限公司>>,成立于深圳多年,有良好的贸易信誉. 长期为各大公司代开电脑发票和各种地税发票;(广告,运输.建筑安装.服务行业). 价格优惠.并以和各大公司长期合作为目的,承诺所开发票可到税务抵扣验证. 欢迎有意着来电咨询. 联系电话;0755-81535924  013926596824 联系人;张永健 深圳市海天实业有限公司
本公司有部分普通发票（商品销售发票）增值税发票及海关代征增值税专用缴款书及其它服务行业发票, 公路、内河运输发票。可以以低税率为贵公司代开，本公司具有内、外贸生意实力，保证我司开具的票据的真实性。 希望可以合作!共同发展!敬侯您的来电洽谈、咨询！ 联系人： 李先生      联系电话：13632588281 如有打扰望谅解，祝商琪。
贵公司财务(经理); 本公司是<<深圳市海天实业有限公司>>,成立于深圳多年,有良好的贸易信誉. 长期为各大公司代开电脑发票和各种地税发票;(广告,运输.建筑安装.服务行业). 价格优惠.并以和各大公司长期合作为目的,承诺所开发票可到税务抵扣验证. 欢迎有意着来电咨询. 联系电话;0755-81535924  013926596824 联系人;张永健 深圳市海天实业有限公司
贵公司经理 你好！ 本公司为广州市荣兴贸易有限公司，本着双方公司利益的出发点可向外代开 普通国（地）税发票，商品销售发票，运输发票，餐饮发票，广告发票，服务行业， 等等．．．．．．普通发票只收２％个税点． 如贵公司在这方面上有需要可来电咨询，欢迎诚意的合作． 联系人：唐娜 联系电话：020-33520954 联系手机：13533999436
贵公司经理 你好！ 本公司为广州市荣兴贸易有限公司，本着双方公司利益的出发点可向外代开 普通国（地）税发票，商品销售发票，运输发票，餐饮发票，广告发票，服务行业， 等等．．．．．．普通发票只收２％个税点． 如贵公司在这方面上有需要可来电咨询，欢迎诚意的合作． 联系人：唐娜 联系电话：020-33520954 联系手机：13533999436
国家企业培训师资格认证考试通知 今年下半年第二期企业培训师资格认证培训班定于10月22日开班，想参加本届国 家统一考试的学员，请在10月份之前准备好相关资料报名。由于人数众多，报名 从速，额满为止，敬请谅解！详情请见：http://www.21renzheng.com/servi ce/hrd.htm 咨询电话：（020）61311042  杨老师 24小时服务热线：13316081650 张老师 [BUTTON]
TOTO卫洗丽 5.3 折 / 美标 5.4 折 数百种建材产品特价销售 更多特色服务： ・免费提供装潢预算 ・装潢专家在线答疑  ・数千张装潢照片可供查询 ・建材淘宝乐，超特价满载            ・数百种建材产品组织团购 欲知更多团购产品的价格，请登陆 www.funasia.cn ！网站还会广大网友提供在线答疑、免费制作装潢预算等帮助！千万别错过这个 机会！
本公司有部分普通发票（商品销售发票）增值税发票及海关代征增值税专用缴款书及其它服务行业发票, 公路、内河运输发票。可以以低税率为贵公司代开，本公司具有内、外贸生意实力，保证我司开具的票据的真实性。 希望可以合作!共同发展!敬侯您的来电洽谈、咨询！ 联系人： 李先生      联系电话：13632588281 如有打扰望谅解，祝商琪。
致贵公司财务： 我司是广州平云贸易有限公司，在广东有一定的经济能力和相当的社会地位， 近来由于业务括展，经研究决定对外代开结余的商品-服务-广告-建筑等一系列发票。 所收税点依所开之数量取决。 本公司宗旨是所开出发票皆可验证后付款，如有需欢迎来电咨询。 联系人：罗先生 联系电话：13828451326 广州平云贸易有限公司
国家企业培训师资格认证考试通知 今年下半年第二期企业培训师资格认证培训班定于10月22日开班，想参加本届国 家统一考试的学员，请在10月份之前准备好相关资料报名。由于人数众多，报名 从速，额满为止，敬请谅解！详情请见：http://www.21renzheng.com/servi ce/hrd.htm 咨询电话：（020）61311042  杨老师 24小时服务热线：13316081650 张老师 [BUTTON]
国家企业培训师资格认证考试通知 今年下半年第二期企业培训师资格认证培训班定于10月22日开班，想参加本届国 家统一考试的学员，请在10月份之前准备好相关资料报名。由于人数众多，报名 从速，额满为止，敬请谅解！详情请见：http://www.21renzheng.com/servi ce/hrd.htm 咨询电话：（020）61311042  杨老师 24小时服务热线：13316081650 张老师 [BUTTON]
TOTO卫洗丽 5.3 折 / 美标 5.4 折 数百种建材产品特价销售 更多特色服务： ・免费提供装潢预算 ・装潢专家在线答疑  ・数千张装潢照片可供查询 ・建材淘宝乐，超特价满载            ・数百种建材产品组织团购 欲知更多团购产品的价格，请登陆 www.funasia.cn ！网站还会广大网友提供在线答疑、免费制作装潢预算等帮助！千万别错过这个 机会！
转让超薄蓝光DVD播放机一台： 可播放DVD，DVCD，SVCD，VCD，CD，HDCD，CD-R，CD-RW，JPEG， MR。OKO，MP3，DVD+RW/R，DVD-RW/R，等光盘 各种接口齐全，S端子，PPY色差，视频输入，AV，铜轴，光钎， 重低音，中置，话筒，左右环绕等，电压110V-250V自适应 随机附送：DVD-8000MB光盘 产品已通过CCC认证，证书编号：2005010806076646执行标准:ISO9002 有意者邮件联系
国家企业培训师资格认证考试通知 今年下半年第二期企业培训师资格认证培训班定于10月22日开班，想参加本届国 家统一考试的学员，请在10月份之前准备好相关资料报名。由于人数众多，报名 从速，额满为止，敬请谅解！详情请见：http://www.21renzheng.com/servi ce/hrd.htm 咨询电话：（020）61311042  杨老师 24小时服务热线：13316081650 张老师 [BUTTON]
新时代健康产业作为中国新时代集团的支柱产业，以“国珍”牌松花粉成功上市为标志，迄今已研制出了以“国珍” 为主品牌的系列健康食品和生活用品，受益者逾万。被商务部评为 AAA级（最高级别）诚心企业。 目前以进入蓄势待发阶段，一个创造财富的黄金时期即将来临。 详情登陆http://www.sab365.com/ Emil:sjygz@sab365.com 电话：010－65531187
真的太好了，这个软件可以合成文字，随便你输入什么内容，他都能够合成，有男声和甜美的女声，朗读速 度可以随便调节。还可以保存合成好的语音文件，以后你看新闻时就不用担心眼睛累了，可以让他来读，你泡 杯茶坐在沙发上听，彻底解放您的眼睛，犒劳您的耳朵。软件的下载地址是http://www.spqj.com/KDVoice/in dex.asp，软件完全免费
网址：www.8wx8.com,电话：0551-3961292，QQ：415455699 2、网站制作（复制，你所见的网站，就是你的网上家园）
、、、、、、、

data_helpers.py

# encoding: UTF-8import numpy as np
import re
import itertools
from collections import Counter
import os
import word2vec_helpers
import time
import pickledef load_data_and_labels(input_text_file, input_label_file, num_labels):x_text = read_and_clean_zh_file(input_text_file)y = None if not os.path.exists(input_label_file) else map(int, list(open(input_label_file, "r").readlines()))return (x_text, y)def load_positive_negative_data_files(positive_data_file, negative_data_file):"""Loads MR polarity data from files, splits the data into words and generates labels.Returns split sentences and labels."""# Load data from filespositive_examples = read_and_clean_zh_file(positive_data_file)negative_examples = read_and_clean_zh_file(negative_data_file)# Combine datax_text = positive_examples + negative_examples# Generate labelspositive_labels = [[0, 1] for _ in positive_examples]negative_labels = [[1, 0] for _ in negative_examples]y = np.concatenate([positive_labels, negative_labels], 0)return [x_text, y]def padding_sentences(input_sentences, padding_token, padding_sentence_length = None):sentences = [sentence.split(' ') for sentence in input_sentences]max_sentence_length = padding_sentence_length if padding_sentence_length is not None else max([len(sentence) for sentence in sentences])for sentence in sentences:if len(sentence) > max_sentence_length:sentence = sentence[:max_sentence_length]else:sentence.extend([padding_token] * (max_sentence_length - len(sentence)))return (sentences, max_sentence_length)def batch_iter(data, batch_size, num_epochs, shuffle=True):'''Generate a batch iterator for a dataset'''data = np.array(data)data_size = len(data)num_batches_per_epoch = int((data_size - 1) / batch_size) + 1for epoch in range(num_epochs):if shuffle:# Shuffle the data at each epochshuffle_indices = np.random.permutation(np.arange(data_size))shuffled_data = data[shuffle_indices]else:shuffled_data = datafor batch_num in range(num_batches_per_epoch):start_idx = batch_num * batch_sizeend_idx = min((batch_num + 1) * batch_size, data_size)yield shuffled_data[start_idx : end_idx]def test():# Test clean_strprint("Test")#print(clean_str("This's a huge dog! Who're going to the top."))# Test load_positive_negative_data_files#x_text,y = load_positive_negative_data_files("./tiny_data/rt-polarity.pos", "./tiny_data/rt-polarity.neg")#print(x_text)#print(y)# Test batch_iter#batches = batch_iter(x_text, 2, 4)#for batch in batches:#    print(batch)def mkdir_if_not_exist(dirpath):if not os.path.exists(dirpath):os.mkdir(dirpath)def seperate_line(line):return ''.join([word + ' ' for word in line])def read_and_clean_zh_file(input_file, output_cleaned_file = None):lines = list(open(input_file, "rb").readlines())lines = [clean_str(seperate_line(line.decode('utf-8'))) for line in lines]if output_cleaned_file is not None:with open(output_cleaned_file, 'w') as f:for line in lines:f.write((line + '\n').encode('utf-8'))return linesdef clean_str(string):string = re.sub(r"[^\u4e00-\u9fff]", " ", string)#string = re.sub(r"[^A-Za-z0-9(),!?\'\`]", " ", string)#string = re.sub(r"\'s", " \'s", string)#string = re.sub(r"\'ve", " \'ve", string)#string = re.sub(r"n\'t", " n\'t", string)#string = re.sub(r"\'re", " \'re", string)#string = re.sub(r"\'d", " \'d", string)#string = re.sub(r"\'ll", " \'ll", string)#string = re.sub(r",", " , ", string)#string = re.sub(r"!", " ! ", string)#string = re.sub(r"\(", " \( ", string)#string = re.sub(r"\)", " \) ", string)#string = re.sub(r"\?", " \? ", string)string = re.sub(r"\s{2,}", " ", string)#return string.strip().lower()return string.strip()def saveDict(input_dict, output_file):with open(output_file, 'wb') as f:pickle.dump(input_dict, f) def loadDict(dict_file):output_dict = Nonewith open(dict_file, 'rb') as f:output_dict = pickle.load(f)return output_dict

text_cnn.py

import tensorflow as tf
import numpy as npclass TextCNN(object):'''A CNN for text classificationUses and embedding layer, followed by a convolutional, max-pooling and softmax layer.'''def __init__(self, sequence_length, num_classes,embedding_size, filter_sizes, num_filters, l2_reg_lambda=0.0):# Placeholders for input, output, dropoutself.input_x = tf.placeholder(tf.float32, [None, sequence_length, embedding_size], name = "input_x")self.input_y = tf.placeholder(tf.float32, [None, num_classes], name = "input_y")self.dropout_keep_prob = tf.placeholder(tf.float32, name = "dropout_keep_prob")# Keeping track of l2 regularization loss (optional)l2_loss = tf.constant(0.0)# Embedding layer# self.embedded_chars = [None(batch_size), sequence_size, embedding_size]# self.embedded_chars = [None(batch_size), sequence_size, embedding_size, 1(num_channels)]self.embedded_chars = self.input_xself.embedded_chars_expended = tf.expand_dims(self.embedded_chars, -1)# Create a convolution + maxpool layer for each filter sizepooled_outputs = []for i, filter_size in enumerate(filter_sizes):with tf.name_scope("conv-maxpool-%s" % filter_size):# Convolution layerfilter_shape = [filter_size, embedding_size, 1, num_filters]W = tf.Variable(tf.truncated_normal(filter_shape, stddev=0.1), name="W")b = tf.Variable(tf.constant(0.1, shape=[num_filters]), name="b")conv = tf.nn.conv2d(self.embedded_chars_expended,W,strides=[1,1,1,1],padding="VALID",name="conv")# Apply nonlinearityh = tf.nn.relu(tf.nn.bias_add(conv, b), name = "relu")# Maxpooling over the outputspooled = tf.nn.max_pool(h,ksize=[1, sequence_length - filter_size + 1, 1, 1],strides=[1,1,1,1],padding="VALID",name="pool")pooled_outputs.append(pooled)# Combine all the pooled featuresnum_filters_total = num_filters * len(filter_sizes)self.h_pool = tf.concat(3, pooled_outputs)self.h_pool_flat = tf.reshape(self.h_pool, [-1, num_filters_total])# Add dropoutwith tf.name_scope("dropout"):self.h_drop = tf.nn.dropout(self.h_pool_flat, self.dropout_keep_prob)# Final (unnomalized) scores and predictionswith tf.name_scope("output"):W = tf.get_variable("W",shape = [num_filters_total, num_classes],initializer = tf.contrib.layers.xavier_initializer())b = tf.Variable(tf.constant(0.1, shape=[num_classes], name = "b"))l2_loss += tf.nn.l2_loss(W)l2_loss += tf.nn.l2_loss(b)self.scores = tf.nn.xw_plus_b(self.h_drop, W, b, name = "scores")self.predictions = tf.argmax(self.scores, 1, name = "predictions")# Calculate Mean cross-entropy losswith tf.name_scope("loss"):losses = tf.nn.softmax_cross_entropy_with_logits(logits = self.scores, labels = self.input_y)self.loss = tf.reduce_mean(losses) + l2_reg_lambda * l2_loss# Accuracywith tf.name_scope("accuracy"):correct_predictions = tf.equal(self.predictions, tf.argmax(self.input_y, 1))self.accuracy = tf.reduce_mean(tf.cast(correct_predictions, "float"), name = "accuracy")

word2vec_helpers.py

# -*- coding: utf-8 -*-'''
python word2vec_helpers.py input_file output_model_file output_vector_file
'''# import modules & set up logging
import os
import sys
import logging
import multiprocessing
import time
import jsonfrom gensim.models import Word2Vec
from gensim.models.word2vec import LineSentencedef output_vocab(vocab):for k, v in vocab.items():print(k)def embedding_sentences(sentences, embedding_size = 128, window = 5, min_count = 5, file_to_load = None, file_to_save = None):if file_to_load is not None:w2vModel = Word2Vec.load(file_to_load)else:w2vModel = Word2Vec(sentences, size = embedding_size, window = window, min_count = min_count, workers = multiprocessing.cpu_count())if file_to_save is not None:w2vModel.save(file_to_save)all_vectors = []embeddingDim = w2vModel.vector_sizeembeddingUnknown = [0 for i in range(embeddingDim)]for sentence in sentences:this_vector = []for word in sentence:if word in w2vModel.wv.vocab:this_vector.append(w2vModel[word])else:this_vector.append(embeddingUnknown)all_vectors.append(this_vector)return all_vectorsdef generate_word2vec_files(input_file, output_model_file, output_vector_file, size = 128, window = 5, min_count = 5):start_time = time.time()# trim unneeded model memory = use(much) less RAM# model.init_sims(replace=True)model = Word2Vec(LineSentence(input_file), size = size, window = window, min_count = min_count, workers = multiprocessing.cpu_count())model.save(output_model_file)model.wv.save_word2vec_format(output_vector_file, binary=False)end_time = time.time()print("used time : %d s" % (end_time - start_time))def run_main():program = os.path.basename(sys.argv[0])logger = logging.getLogger(program)logging.basicConfig(format='%(asctime)s : %(levelname)s : %(message)s', level=logging.INFO)logger.info("running %s" % ' '.join(sys.argv))# check and process input argumentsif len(sys.argv) < 4:print (globals()['__doc__'] % locals())sys.exit(1)input_file, output_model_file, output_vector_file = sys.argv[1:4]generate_word2vec_files(input_file, output_model_file, output_vector_file) def test():vectors = embedding_sentences([['first', 'sentence'], ['second', 'sentence']], embedding_size = 4, min_count = 1)print(vectors)

train.py

#! /usr/bin/env python
# encoding: utf-8import tensorflow as tf
import numpy as np
import os
import time
import datetime
import data_helpers
import word2vec_helpers
from text_cnn import TextCNN# Parameters
# =======================================================# Data loading parameters
tf.flags.DEFINE_float("dev_sample_percentage", .1, "Percentage of the training data to use for validation")
#tf.flags.DEFINE_string("positive_data_file", "./data/rt-polaritydata/rt-polarity.pos", "Data source for the positive data.")
#tf.flags.DEFINE_string("negative_data_file", "./data/rt-polaritydata/rt-polarity.neg", "Data source for the negative data.")
tf.flags.DEFINE_string("positive_data_file", "./data/ham_5000.utf8", "Data source for the positive data.")
tf.flags.DEFINE_string("negative_data_file", "./data/spam_5000.utf8", "Data source for the negative data.")
tf.flags.DEFINE_integer("num_labels", 2, "Number of labels for data. (default: 2)")# Model hyperparameters
tf.flags.DEFINE_integer("embedding_dim", 128, "Dimensionality of character embedding (default: 128)")
tf.flags.DEFINE_string("filter_sizes", "3,4,5", "Comma-spearated filter sizes (default: '3,4,5')")
tf.flags.DEFINE_integer("num_filters", 128, "Number of filters per filter size (default: 128)")
tf.flags.DEFINE_float("dropout_keep_prob", 0.5, "Dropout keep probability (default: 0.5)")
tf.flags.DEFINE_float("l2_reg_lambda", 0.0, "L2 regularization lambda (default: 0.0)")# Training paramters
tf.flags.DEFINE_integer("batch_size", 64, "Batch Size (default: 64)")
tf.flags.DEFINE_integer("num_epochs", 200, "Number of training epochs (default: 200)")
tf.flags.DEFINE_integer("evaluate_every", 100, "Evalue model on dev set after this many steps (default: 100)")
tf.flags.DEFINE_integer("checkpoint_every", 100, "Save model after this many steps (defult: 100)")
tf.flags.DEFINE_integer("num_checkpoints", 5, "Number of checkpoints to store (default: 5)")# Misc parameters
tf.flags.DEFINE_boolean("allow_soft_placement", True, "Allow device soft device placement")
tf.flags.DEFINE_boolean("log_device_placement", False, "Log placement of ops on devices")# Parse parameters from commands
FLAGS = tf.flags.FLAGS
FLAGS._parse_flags()
print("\nParameters:")
for attr, value in sorted(FLAGS.__flags.items()):print("{}={}".format(attr.upper(), value))
print("")# Prepare output directory for models and summaries
# =======================================================timestamp = str(int(time.time()))
out_dir = os.path.abspath(os.path.join(os.path.curdir, "runs", timestamp))
print("Writing to {}\n".format(out_dir))
if not os.path.exists(out_dir):os.makedirs(out_dir)# Data preprocess
# =======================================================# Load data
print("Loading data...")
x_text, y = data_helpers.load_positive_negative_data_files(FLAGS.positive_data_file, FLAGS.negative_data_file)# Get embedding vector
sentences, max_document_length = data_helpers.padding_sentences(x_text, '<PADDING>')
x = np.array(word2vec_helpers.embedding_sentences(sentences, embedding_size = FLAGS.embedding_dim, file_to_save = os.path.join(out_dir, 'trained_word2vec.model')))
print("x.shape = {}".format(x.shape))
print("y.shape = {}".format(y.shape))# Save params
training_params_file = os.path.join(out_dir, 'training_params.pickle')
params = {'num_labels' : FLAGS.num_labels, 'max_document_length' : max_document_length}
data_helpers.saveDict(params, training_params_file)# Shuffle data randomly
np.random.seed(10)
shuffle_indices = np.random.permutation(np.arange(len(y)))
x_shuffled = x[shuffle_indices]
y_shuffled = y[shuffle_indices]# Split train/test set
# TODO: This is very crude, should use cross-validation
dev_sample_index = -1 * int(FLAGS.dev_sample_percentage * float(len(y)))
x_train, x_dev = x_shuffled[:dev_sample_index], x_shuffled[dev_sample_index:]
y_train, y_dev = y_shuffled[:dev_sample_index], y_shuffled[dev_sample_index:]
print("Train/Dev split: {:d}/{:d}".format(len(y_train), len(y_dev)))# Training
# =======================================================with tf.Graph().as_default():session_conf = tf.ConfigProto(allow_soft_placement = FLAGS.allow_soft_placement,log_device_placement = FLAGS.log_device_placement)sess = tf.Session(config = session_conf)with sess.as_default():cnn = TextCNN(sequence_length = x_train.shape[1],num_classes = y_train.shape[1],embedding_size = FLAGS.embedding_dim,filter_sizes = list(map(int, FLAGS.filter_sizes.split(","))),num_filters = FLAGS.num_filters,l2_reg_lambda = FLAGS.l2_reg_lambda)# Define Training procedureglobal_step = tf.Variable(0, name="global_step", trainable=False)optimizer = tf.train.AdamOptimizer(1e-3)grads_and_vars = optimizer.compute_gradients(cnn.loss)train_op = optimizer.apply_gradients(grads_and_vars, global_step=global_step)# Keep track of gradient values and sparsity (optional)grad_summaries = []for g, v in grads_and_vars:if g is not None:grad_hist_summary = tf.summary.histogram("{}/grad/hist".format(v.name), g)sparsity_summary = tf.summary.scalar("{}/grad/sparsity".format(v.name), tf.nn.zero_fraction(g))grad_summaries.append(grad_hist_summary)grad_summaries.append(sparsity_summary)grad_summaries_merged = tf.summary.merge(grad_summaries)# Output directory for models and summariesprint("Writing to {}\n".format(out_dir))# Summaries for loss and accuracyloss_summary = tf.summary.scalar("loss", cnn.loss)acc_summary = tf.summary.scalar("accuracy", cnn.accuracy)# Train Summariestrain_summary_op = tf.summary.merge([loss_summary, acc_summary, grad_summaries_merged])train_summary_dir = os.path.join(out_dir, "summaries", "train")train_summary_writer = tf.summary.FileWriter(train_summary_dir, sess.graph)# Dev summariesdev_summary_op = tf.summary.merge([loss_summary, acc_summary])dev_summary_dir = os.path.join(out_dir, "summaries", "dev")dev_summary_writer = tf.summary.FileWriter(dev_summary_dir, sess.graph)# Checkpoint directory. Tensorflow assumes this directory already exists so we need to create itcheckpoint_dir = os.path.abspath(os.path.join(out_dir, "checkpoints"))checkpoint_prefix = os.path.join(checkpoint_dir, "model")if not os.path.exists(checkpoint_dir):os.makedirs(checkpoint_dir)saver = tf.train.Saver(tf.global_variables(), max_to_keep=FLAGS.num_checkpoints)# Initialize all variablessess.run(tf.global_variables_initializer())def train_step(x_batch, y_batch):"""A single training step"""feed_dict = {cnn.input_x: x_batch,cnn.input_y: y_batch,cnn.dropout_keep_prob: FLAGS.dropout_keep_prob}_, step, summaries, loss, accuracy = sess.run([train_op, global_step, train_summary_op, cnn.loss, cnn.accuracy],feed_dict)time_str = datetime.datetime.now().isoformat()print("{}: step {}, loss {:g}, acc {:g}".format(time_str, step, loss, accuracy))train_summary_writer.add_summary(summaries, step)def dev_step(x_batch, y_batch, writer=None):"""Evaluates model on a dev set"""feed_dict = {cnn.input_x: x_batch,cnn.input_y: y_batch,cnn.dropout_keep_prob: 1.0}step, summaries, loss, accuracy = sess.run([global_step, dev_summary_op, cnn.loss, cnn.accuracy],feed_dict)time_str = datetime.datetime.now().isoformat()print("{}: step {}, loss {:g}, acc {:g}".format(time_str, step, loss, accuracy))if writer:writer.add_summary(summaries, step)# Generate batchesbatches = data_helpers.batch_iter(list(zip(x_train, y_train)), FLAGS.batch_size, FLAGS.num_epochs)# Training loop. For each batch...for batch in batches:x_batch, y_batch = zip(*batch)train_step(x_batch, y_batch)current_step = tf.train.global_step(sess, global_step)if current_step % FLAGS.evaluate_every == 0:print("\nEvaluation:")dev_step(x_dev, y_dev, writer=dev_summary_writer)print("")if current_step % FLAGS.checkpoint_every == 0:path = saver.save(sess, checkpoint_prefix, global_step=current_step)print("Saved model checkpoint to {}\n".format(path))

eval.py

#! /usr/bin/env pythonimport tensorflow as tf
import numpy as np
import os
import time
import datetime
import data_helpers
import word2vec_helpers
from text_cnn import TextCNN
import csv# Parameters
# ==================================================# Data Parameters
tf.flags.DEFINE_string("input_text_file", "./data/spam_100.utf8", "Test text data source to evaluate.")
tf.flags.DEFINE_string("input_label_file", "", "Label file for test text data source.")# Eval Parameters
tf.flags.DEFINE_integer("batch_size", 64, "Batch Size (default: 64)")
tf.flags.DEFINE_string("checkpoint_dir", "", "Checkpoint directory from training run")
tf.flags.DEFINE_boolean("eval_train", True, "Evaluate on all training data")# Misc Parameters
tf.flags.DEFINE_boolean("allow_soft_placement", True, "Allow device soft device placement")
tf.flags.DEFINE_boolean("log_device_placement", False, "Log placement of ops on devices")FLAGS = tf.flags.FLAGS
FLAGS._parse_flags()
print("\nParameters:")
for attr, value in sorted(FLAGS.__flags.items()):print("{}={}".format(attr.upper(), value))
print("")# validate
# ==================================================# validate checkout point file
checkpoint_file = tf.train.latest_checkpoint(FLAGS.checkpoint_dir)
if checkpoint_file is None:print("Cannot find a valid checkpoint file!")exit(0)
print("Using checkpoint file : {}".format(checkpoint_file))# validate word2vec model file
trained_word2vec_model_file = os.path.join(FLAGS.checkpoint_dir, "..", "trained_word2vec.model")
if not os.path.exists(trained_word2vec_model_file):print("Word2vec model file \'{}\' doesn't exist!".format(trained_word2vec_model_file))
print("Using word2vec model file : {}".format(trained_word2vec_model_file))# validate training params file
training_params_file = os.path.join(FLAGS.checkpoint_dir, "..", "training_params.pickle")
if not os.path.exists(training_params_file):print("Training params file \'{}\' is missing!".format(training_params_file))
print("Using training params file : {}".format(training_params_file))# Load params
params = data_helpers.loadDict(training_params_file)
num_labels = int(params['num_labels'])
max_document_length = int(params['max_document_length'])# Load data
if FLAGS.eval_train:x_raw, y_test = data_helpers.load_data_and_labels(FLAGS.input_text_file, FLAGS.input_label_file, num_labels)
else:x_raw = ["a masterpiece four years in the making", "everything is off."]y_test = [1, 0]# Get Embedding vector x_test
sentences, max_document_length = data_helpers.padding_sentences(x_raw, '<PADDING>', padding_sentence_length = max_document_length)
x_test = np.array(word2vec_helpers.embedding_sentences(sentences, file_to_load = trained_word2vec_model_file))
print("x_test.shape = {}".format(x_test.shape))# Evaluation
# ==================================================
print("\nEvaluating...\n")
checkpoint_file = tf.train.latest_checkpoint(FLAGS.checkpoint_dir)
graph = tf.Graph()
with graph.as_default():session_conf = tf.ConfigProto(allow_soft_placement=FLAGS.allow_soft_placement,log_device_placement=FLAGS.log_device_placement)sess = tf.Session(config=session_conf)with sess.as_default():# Load the saved meta graph and restore variablessaver = tf.train.import_meta_graph("{}.meta".format(checkpoint_file))saver.restore(sess, checkpoint_file)# Get the placeholders from the graph by nameinput_x = graph.get_operation_by_name("input_x").outputs[0]# input_y = graph.get_operation_by_name("input_y").outputs[0]dropout_keep_prob = graph.get_operation_by_name("dropout_keep_prob").outputs[0]# Tensors we want to evaluatepredictions = graph.get_operation_by_name("output/predictions").outputs[0]# Generate batches for one epochbatches = data_helpers.batch_iter(list(x_test), FLAGS.batch_size, 1, shuffle=False)# Collect the predictions hereall_predictions = []for x_test_batch in batches:batch_predictions = sess.run(predictions, {input_x: x_test_batch, dropout_keep_prob: 1.0})all_predictions = np.concatenate([all_predictions, batch_predictions])# Print accuracy if y_test is defined
if y_test is not None:correct_predictions = float(sum(all_predictions == y_test))print("Total number of test examples: {}".format(len(y_test)))print("Accuracy: {:g}".format(correct_predictions/float(len(y_test))))# Save the evaluation to a csv
predictions_human_readable = np.column_stack((np.array([text.encode('utf-8') for text in x_raw]), all_predictions))
out_path = os.path.join(FLAGS.checkpoint_dir, "..", "prediction.csv")
print("Saving evaluation to {0}".format(out_path))
with open(out_path, 'w') as f:csv.writer(f).writerows(predictions_human_readable)

基于Tensorflow里CNN文本分类相关推荐

1. [深度学习]-基于tensorflow的CNN和RNN-LSTM文本情感分析对比

   基于tensorflow的CNN和LSTM文本情感分析对比 1. 背景介绍 2. 数据集介绍 2.0 wordsList.npy 2.1 wordVectors.npy 2.2 idsMatrix.n ...

2. 基于TensorFlow的CNN卷积网络模型花卉分类GUI版（2）

   一.项目描述 10类花的图片1100张,按{牡丹,月季,百合,菊花,荷花,紫荆花,梅花,-}标注,其中1000张作为训练样本,100张作为测试样本,设计一个CNN卷积神经网络花卉分类器进行花卉的分类, ...

3. 在TensorFlow中实现文本分类的CNN

   在TensorFlow中实现文本分类的CNN 在TensorFlow中实现文本分类的CNN 数据和预处理 模型 实现 1 输入占位符 2 向量层 3 卷积层和池化层 4 Dropout 层 5 得分和 ...

4. 自然语言处理入门实战3：基于深度学习的文本分类（2）

   基于深度学习的文本分类(2) 数据集 数据预处理 CNN模型 RNN模型 利用CNN模型进行训练和测试 利用RNN模型进行训练和测试 预测 总结 参考 本文主要是使用CNN和RNN进行文本分类操作. ...

5. Datawhale NLP入门：Task5 基于深度学习的文本分类2

   Task5 基于深度学习的文本分类2 在上一章节,我们通过FastText快速实现了基于深度学习的文本分类模型,但是这个模型并不是最优的.在本章我们将继续深入. 基于深度学习的文本分类 本章将继续学习 ...

6. Task5 基于深度学习的文本分类2

   Task5 基于深度学习的文本分类2 在上一章节,我们通过FastText快速实现了基于深度学习的文本分类模型,但是这个模型并不是最优的.在本章我们将继续深入. 基于深度学习的文本分类 本章将继续学习 ...

7. Datawhale零基础入门NLP day5/Task5基于深度学习的文本分类2

   基于深度学习的文本分类 本章将继续学习基于深度学习的文本分类. 学习目标 学习Word2Vec的使用和基础原理 学习使用TextCNN.TextRNN进行文本表示 学习使用HAN网络结构完成文本分类 ...

8. Datawhale零基础入门NLP赛事 - Task5 基于深度学习的文本分类2

   在上一章节,我们通过FastText快速实现了基于深度学习的文本分类模型,但是这个模型并不是最优的.在本章我们将继续深入. 基于深度学习的文本分类 本章将继续学习基于深度学习的文本分类. 学习目标 学 ...

9. 新闻文本分类--任务5 基于深度学习的文本分类2

   Task5 基于深度学习的文本分类2 在上一章节,我们通过FastText快速实现了基于深度学习的文本分类模型,但是这个模型并不是最优的.在本章我们将继续深入. 基于深度学习的文本分类 本章将继续学习 ...

最新文章

1. [转]笑话: 耐力惊人的三只乌龟
2. oracle终止用户会话
3. 机器学习算法学习---模型融合和提升的算法（五）
4. ［图］为C# Windows服务添加安装程序
5. 深入理解分布式技术 - 分布式调用跟踪
6. 【已解决】CMake Error: Cannot determine link language for target “xxx“. CMake Error: CMake can not determ
7. Eclipse如何提高开发效率
8. linux下搭建git服务器
9. Junit5集成到SpringBoot工程
10. linux下mysql启动和关闭
11. web安全day13：简单深透测试流程
12. .fnt 字体不能正常显示
13. 新品流量 DRS动态评分 店铺层级 搜索权重 增加流量 保持流量持续上升的技巧
14. Freemarker函数
15. 案例：Java用面向对象的思想设计游戏中的角色
16. 实现两直角坐标系转换
17. 2019-9-11-数据结构查找方法总结
18. 关于Google Map 叠加层之Polyline(折线)、Polygon(多边形)、InfoWindow(信息窗口)
19. python matplotlib绘制 3D图像专题 （三维柱状图、曲面图、散点图、曲线图合集）
20. 数字调制系列：IQ调制基本理论

热门文章

1. 什么是数据驱动测试？学习创建框架
2. 【从面试出发学习java】- 缓存 - Redis面试题
3. Windows下的MySQL实例没有mysql.user表#Olivia丶长歌#
4. 解决conda install pkgs found conflict问题
5. php如何配置gii,PHP Framework YII的里的gii设置。
6. ubuntu新硬盘创建分区步骤
7. 计算机可以计算出十的一百次方吗,世界上最大的数字单位 古戈尔(1古戈尔等于10的100次方)...
8. 以太坊概念知识入门篇 1
9. Android监听前后台切换展示开屏广告
10. iOS 11种键盘布局总结