PyTorch入门-自然语言分类任务
一. 情感分析
1. 准备数据
TorchText中的一个重要概念是Field。Field决定了你的数据会被怎样处理。在我们的情感分类任务中，我们所需要接触到的数据有文本字符串和两种情感，“pos"或者"neg”。
Field的参数制定了数据会被怎样处理。
使用TEXT field来定义如何处理电影评论，使用LABEL field来处理两个情感类别。
TEXT field带有tokenize=‘spacy’，这表示我们会用spaCy tokenizer来tokenize英文句子。如果我们不特别声明tokenize这个参数，那么默认的分词方法是使用空格。
安装spaCy
pip install -U spacy
python -m spacy download enLABEL由LabelField定义。这是一种特别的用来处理label的Field。
import torch
from torchtext import dataSEED = 1234torch.manual_seed(SEED)
torch.cuda.manual_seed(SEED)
torch.backends.cudnn.deterministic = TrueTEXT = data.Field(tokenize='spacy')
LABEL = data.LabelField(dtype=torch.float)
说明：
- TorchText支持很多常见的自然语言处理数据集。
- 下面的代码会自动下载IMDb数据集，然后分成train/test两个torchtext.datasets类别。数据被前面的Fields处理。IMDb数据集一共有50000电影评论，每个评论都被标注为正面的或负面的。
from torchtext import datasets
train_data, test_data = datasets.IMDB.splits(TEXT, LABEL)
查看每个数据split有多少条数据
print(f'Number of training examples: {len(train_data)}')
print(f'Number of testing examples: {len(test_data)}')
结果：
Number of training examples: 25000
Number of testing examples: 25000
查看一个example
print(vars(train_data.examples[0]))
结果：
{‘text’: [‘Brilliant’, ‘adaptation’, ‘of’, ‘the’, ‘novel’, ‘that’, ‘made’, ‘famous’, ‘the’, ‘relatives’, ‘of’, ‘Chilean’, ‘President’, ‘Salvador’, ‘Allende’, ‘killed’, ‘.’, ‘In’, ‘the’, ‘environment’, ‘of’, ‘a’, ‘large’, ‘estate’, ‘that’, ‘arises’, ‘from’, ‘the’, ‘ruins’, ‘,’, ‘becoming’, ‘a’, ‘force’, ‘to’, ‘abuse’, ‘and’, ‘exploitation’, ‘of’, ‘outrage’, ‘,’, ‘a’, ‘luxury’, ‘estate’, ‘for’, ‘the’, ‘benefit’, ‘of’, ‘the’, ‘upstart’, ‘Esteban’, ‘Trueba’, ‘and’, ‘his’, ‘undeserved’, ‘family’, ‘,’, ‘the’, ‘brilliant’, ‘Danish’, ‘director’, ‘Bille’, ‘August’, ‘recreates’, ‘,’, ‘in’, ‘micro’, ‘,’, ‘which’, ‘at’, ‘the’, ‘time’, ‘would’, ‘be’, ‘the’, ‘process’, ‘leading’, ‘to’, ‘the’, ‘greatest’, ‘infamy’, ‘of’, ‘his’, ‘story’, ‘to’, ‘the’, ‘hardened’, ‘Chilean’, ‘nation’, ‘,’, ‘and’, ‘whose’, ‘main’, ‘character’, ‘would’, ‘Augusto’, ‘Pinochet’, ‘(’, ‘Stephen’, ‘similarities’, ‘with’, ‘it’, ‘are’, ‘inevitable’, ‘:’, ‘recall’, ‘,’, ‘as’, ‘an’, ‘example’, ‘,’, ‘that’, ‘image’, ‘of’, ‘the’, ‘senator’, ‘with’, ‘dark’, ‘glasses’, ‘that’, ‘makes’, ‘him’, ‘the’, ‘wink’, ‘to’, ‘the’, ‘general’, ‘to’, ‘begin’, ‘making’, ‘the’, ‘palace).<br’, ‘/><br’, ‘/>Bille’, ‘August’, ‘attends’, ‘an’, ‘exceptional’, ‘cast’, ‘in’, ‘the’, ‘Jeremy’, ‘protruding’, ‘Irons’, ‘,’, ‘whose’, ‘character’, ‘changes’, ‘from’, ‘arrogance’, ‘and’, ‘extreme’, ‘cruelty’, ‘,’, ‘the’, ‘hard’, ‘lesson’, ‘that’, ‘life’, ‘always’, ‘brings’, ‘us’, ‘to’, ‘almost’, ‘force’, ‘us’, ‘to’, ‘change’, ‘.’, ‘In’, ‘Esteban’, ‘fully’, ‘applies’, ‘the’, ‘law’, ‘of’, ‘resonance’, ‘,’, ‘with’, ‘great’, ‘wisdom’, ‘,’, ‘Solomon’, ‘describes’, ‘in’, ‘these’, ‘words:"The’, ‘things’, ‘that’, ‘freckles’, ‘are’, ‘the’, ‘same’, ‘punishment’, ‘that’, ‘will’, ‘serve’, ‘you’, ‘.’, ‘"’, ‘<’, ‘br’, ‘/><br’, ‘/>Unforgettable’, ‘Glenn’, ‘Close’, ‘playing’, ‘splint’, ‘,’, ‘the’, ‘tainted’, ‘sister’, ‘of’, ‘Stephen’, ‘,’, ‘whose’, ‘sin’, ‘,’, ‘driven’, ‘by’, ‘loneliness’, ‘,’, ‘spiritual’, ‘and’, ‘platonic’, ‘love’, ‘was’, ‘the’, ‘wife’, ‘of’, ‘his’, ‘cruel’, ‘snowy’, ‘brother’, ‘.’, ‘Meryl’, ‘Streep’, ‘also’, ‘brilliant’, ‘,’, ‘a’, ‘woman’, ‘whose’, ‘name’, ‘came’, ‘to’, ‘him’, ‘like’, ‘a’, ‘glove’, ‘Clara’, ‘.’, ‘With’, ‘telekinetic’, ‘powers’, ‘,’, ‘cognitive’, ‘and’, ‘mediumistic’, ‘,’, ‘this’, ‘hardened’, ‘woman’, ‘,’, ‘loyal’, ‘to’, ‘his’, ‘blunt’, ‘,’, ‘conservative’, ‘husband’, ‘,’, ‘is’, ‘an’, ‘indicator’, ‘of’, ‘character’, ‘and’, ‘self’, ‘-’, ‘control’, ‘that’, ‘we’, ‘wish’, ‘for’, ‘ourselves’, ‘and’, ‘for’, ‘all’, ‘human’, ‘beings’, ‘.’, ‘<’, ‘br’, ‘/><br’, ‘/>Every’, ‘character’, ‘is’, ‘a’, ‘portrait’, ‘of’, ‘virtuosity’, ‘(’, ‘as’, ‘Blanca’, ‘worthy’, ‘rebel’, ‘leader’, ‘Pedro’, ‘Segundo’, ‘unhappy’, ‘…’, ‘)’, ‘or’, ‘a’, ‘portrait’, ‘of’, ‘humiliation’, ‘,’, ‘like’, ‘Stephen’, ‘Jr.’, ‘,’, ‘the’, ‘bastard’, ‘child’, ‘of’, ‘Senator’, ‘,’, ‘who’, ‘serves’, ‘as’, ‘an’, ‘instrument’, ‘for’, ‘the’, ‘return’, ‘of’, ‘the’, ‘boomerang’, ‘.’, ‘<’, ‘br’, ‘/><br’, ‘/>The’, ‘film’, ‘moves’, ‘the’, ‘bowels’, ‘,’, ‘we’, ‘recreated’, ‘some’, ‘facts’, ‘that’, ‘should’, ‘not’, ‘ever’, ‘be’, ‘repeated’, ‘,’, ‘but’, ‘that’, ‘absurdly’, ‘still’, ‘happen’, ‘(’, ‘Colombia’, ‘is’, ‘a’, ‘sad’, ‘example’, ‘)’, ‘and’, ‘another’, ‘reminder’, ‘that’, ‘,’, ‘against’, ‘all’, ‘,’, ‘life’, ‘is’, ‘wonderful’, ‘because’, ‘there’, ‘are’, ‘always’, ‘people’, ‘like’, ‘Isabel’, ‘Allende’, ‘and’, ‘immortalize’, ‘just’, ‘Bille’, ‘August’, ‘.’], ‘label’: ‘pos’}
说明：
- 由于现在只有train/test这两个分类，所以需要创建一个新的validation set。可以使用.split()创建新的分类。
- 默认的数据分割是 70、30，如果我们声明split_ratio，可以改变split之间的比例，split_ratio=0.8表示80%的数据是训练集，20%是验证集。
- 还需要声明random_state这个参数，确保我们每次分割的数据集都是一样的。
import random
train_data, valid_data = train_data.split(random_state=random.seed(SEED))
检查一下现在每个部分有多少条数据
print(f'Number of training examples: {len(train_data)}')
print(f'Number of validation examples: {len(valid_data)}')
print(f'Number of testing examples: {len(test_data)}')
结果：
Number of training examples: 17500
Number of validation examples: 7500
Number of testing examples: 25000
说明：
- 下一步需要创建 vocabulary 。vocabulary 就是把每个单词一一映射到一个数字。
- 用最常见的25k个单词来构建我们的单词表，用max_size这个参数可以做到这一点。
- 所有其他的单词都用来表示。
# TEXT.build_vocab(train_data, max_size=25000)
# LABEL.build_vocab(train_data)
TEXT.build_vocab(train_data, max_size=25000, vectors="glove.6B.100d", unk_init=torch.Tensor.normal_)
LABEL.build_vocab(train_data)
print(f"Unique tokens in TEXT vocabulary: {len(TEXT.vocab)}")
print(f"Unique tokens in LABEL vocabulary: {len(LABEL.vocab)}")
结果：
Unique tokens in TEXT vocabulary: 25002
Unique tokens in LABEL vocabulary: 2
说明：
- 当我们把句子传进模型的时候，我们是按照一个个 batch 穿进去的，也就是说，我们一次传入了好几个句子，而且每个batch中的句子必须是相同的长度。为了确保句子的长度相同，TorchText会把短的句子pad到和最长的句子等长。
训练数据集中最常见的单词
print(TEXT.vocab.freqs.most_common(20))
结果：
[('the', 201455), (',', 192552), ('.', 164402), ('a', 108963), ('and', 108649), ('of', 100010), ('to', 92873), ('is', 76046), ('in', 60904), ('I', 54486), ('it', 53405), ('that', 49155), ('"', 43890), ("'s", 43151), ('this', 42454), ('-', 36769), ('/><br', 35511), ('was', 34990), ('as', 30324), ('with', 29691)]
可以直接用 stoi(string to int) 或者 itos (int to string) 来查看单词表
print(TEXT.vocab.itos[:10])
结果：
['<unk>', '<pad>', 'the', ',', '.', 'a', 'and', 'of', 'to', 'is']
查看labels
print(LABEL.vocab.stoi)
结果：
defaultdict(<function _default_unk_index at 0x7fbec39a79d8>, {'neg': 0, 'pos': 1})
说明：
- 最后一步数据的准备是创建iterators。每个itartion都会返回一个batch的examples。
- 使用BucketIterator。BucketIterator会把长度差不多的句子放到同一个batch中，确保每个batch中不出现太多的padding。
- 严格来说，我们这份notebook中的模型代码都有一个问题，也就是我们把也当做了模型的输入进行训练。更好的做法是在模型中把由产生的输出给消除掉。在这节课中我们简单处理，直接把也用作模型输入了。由于数量不多，模型的效果也不差。
- 如果有GPU，还可以指定每个iteration返回的tensor都在GPU上。
BATCH_SIZE = 64device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')train_iterator, valid_iterator, test_iterator = data.BucketIterator.splits((train_data, valid_data, test_data), batch_size=BATCH_SIZE,device=device)
2. Word Averaging模型
- 首先介绍一个简单的Word Averaging模型。我们把每个单词都通过Embedding层投射成word embedding vector，然后把一句话中的所有word vector做个平均，就是整个句子的vector表示了。接下来把这个sentence vector传入一个Linear层，做分类即可。
- 使用avg_pool2d来做average pooling。目标是把sentence length那个维度平均成1，然后保留embedding这个维度。
- avg_pool2d的kernel size是 (embedded.shape[1], 1)，所以句子长度的那个维度会被压扁。
import torch.nn as nn
import torch.nn.functional as Fclass WordAVGModel(nn.Module):def __init__(self, vocab_size, embedding_dim, output_dim, pad_idx):super().__init__()self.embedding = nn.Embedding(vocab_size, embedding_dim, padding_idx=pad_idx)self.fc = nn.Linear(embedding_dim, output_dim)def forward(self, text):embedded = self.embedding(text) # [sent len, batch size, emb dim]embedded = embedded.permute(1, 0, 2) # [batch size, sent len, emb dim]pooled = F.avg_pool2d(embedded, (embedded.shape[1], 1)).squeeze(1) # [batch size, embedding_dim]return self.fc(pooled)
INPUT_DIM = len(TEXT.vocab)
EMBEDDING_DIM = 100
OUTPUT_DIM = 1
PAD_IDX = TEXT.vocab.stoi[TEXT.pad_token]model = WordAVGModel(INPUT_DIM, EMBEDDING_DIM, OUTPUT_DIM, PAD_IDX)
def count_parameters(model):return sum(p.numel() for p in model.parameters() if p.requires_grad)print(f'The model has {count_parameters(model):,} trainable parameters')
pretrained_embeddings = TEXT.vocab.vectors
model.embedding.weight.data.copy_(pretrained_embeddings)
结果：
tensor([[-0.1117, -0.4966, 0.1631, ..., 1.2647, -0.2753, -0.1325],[-0.8555, -0.7208, 1.3755, ..., 0.0825, -1.1314, 0.3997],[-0.0382, -0.2449, 0.7281, ..., -0.1459, 0.8278, 0.2706],...,[-0.7244, -0.0186, 0.0996, ..., 0.0045, -1.0037, 0.6646],[-1.1243, 1.2040, -0.6489, ..., -0.7526, 0.5711, 1.0081],[ 0.0860, 0.1367, 0.0321, ..., -0.5542, -0.4557, -0.0382]])
（1）训练模型
import torch.optim as optim
optimizer = optim.Adam(model.parameters())
criterion = nn.BCEWithLogitsLoss()
model = model.to(device)
criterion = criterion.to(device)
计算预测的准确率
def binary_accuracy(preds, y):"""Returns accuracy per batch, i.e. if you get 8/10 right, this returns 0.8, NOT 8"""#round predictions to the closest integerrounded_preds = torch.round(torch.sigmoid(preds))correct = (rounded_preds == y).float() #convert into float for divisionacc = correct.sum()/len(correct)return acc
def train(model, iterator, optimizer, criterion):epoch_loss = 0epoch_acc = 0model.train()for batch in iterator:optimizer.zero_grad()predictions = model(batch.text).squeeze(1)loss = criterion(predictions, batch.label)acc = binary_accuracy(predictions, batch.label)loss.backward()optimizer.step()epoch_loss += loss.item()epoch_acc += acc.item()return epoch_loss / len(iterator), epoch_acc / len(iterator)
def evaluate(model, iterator, criterion):epoch_loss = 0epoch_acc = 0model.eval()with torch.no_grad():for batch in iterator:predictions = model(batch.text).squeeze(1)loss = criterion(predictions, batch.label)acc = binary_accuracy(predictions, batch.label)epoch_loss += loss.item()epoch_acc += acc.item()return epoch_loss / len(iterator), epoch_acc / len(iterator)
import timedef epoch_time(start_time, end_time):elapsed_time = end_time - start_timeelapsed_mins = int(elapsed_time / 60)elapsed_secs = int(elapsed_time - (elapsed_mins * 60))return elapsed_mins, elapsed_secs
N_EPOCHS = 10best_valid_loss = float('inf')for epoch in range(N_EPOCHS):start_time = time.time()train_loss, train_acc = train(model, train_iterator, optimizer, criterion)valid_loss, valid_acc = evaluate(model, valid_iterator, criterion)end_time = time.time()epoch_mins, epoch_secs = epoch_time(start_time, end_time)if valid_loss < best_valid_loss:best_valid_loss = valid_losstorch.save(model.state_dict(), 'wordavg-model.pt')print(f'Epoch: {epoch+1:02} | Epoch Time: {epoch_mins}m {epoch_secs}s')print(f'\tTrain Loss: {train_loss:.3f} | Train Acc: {train_acc*100:.2f}%')print(f'\t Val. Loss: {valid_loss:.3f} | Val. Acc: {valid_acc*100:.2f}%')
结果：
Epoch: 01 | Epoch Time: 0m 2sTrain Loss: 0.685 | Train Acc: 56.84%Val. Loss: 0.622 | Val. Acc: 71.09%
Epoch: 02 | Epoch Time: 0m 2sTrain Loss: 0.642 | Train Acc: 71.31%Val. Loss: 0.510 | Val. Acc: 75.48%
Epoch: 03 | Epoch Time: 0m 2sTrain Loss: 0.573 | Train Acc: 78.31%Val. Loss: 0.449 | Val. Acc: 79.52%
Epoch: 04 | Epoch Time: 0m 2sTrain Loss: 0.503 | Train Acc: 82.78%Val. Loss: 0.419 | Val. Acc: 82.72%
Epoch: 05 | Epoch Time: 0m 2sTrain Loss: 0.440 | Train Acc: 85.84%Val. Loss: 0.408 | Val. Acc: 84.75%
Epoch: 06 | Epoch Time: 0m 2sTrain Loss: 0.389 | Train Acc: 87.59%Val. Loss: 0.413 | Val. Acc: 86.02%
Epoch: 07 | Epoch Time: 0m 2sTrain Loss: 0.352 | Train Acc: 88.85%Val. Loss: 0.425 | Val. Acc: 86.92%
Epoch: 08 | Epoch Time: 0m 2sTrain Loss: 0.320 | Train Acc: 89.93%Val. Loss: 0.440 | Val. Acc: 87.54%
Epoch: 09 | Epoch Time: 0m 2sTrain Loss: 0.294 | Train Acc: 90.74%Val. Loss: 0.456 | Val. Acc: 88.09%
Epoch: 10 | Epoch Time: 0m 2sTrain Loss: 0.274 | Train Acc: 91.27%Val. Loss: 0.468 | Val. Acc: 88.49%
3. RNN模型
- 下面尝试把模型换成一个recurrent neural network (RNN)。RNN经常会被用来encode一个sequence [Math Processing Error]
- 使用最后一个hidden state [Math Processing Error]来表示整个句子。
- 然后我们把[Math Processing Error]通过一个线性变换[Math Processing Error]，然后用来预测句子的情感。
class RNN(nn.Module):def __init__(self, vocab_size, embedding_dim, hidden_dim, output_dim, n_layers, bidirectional, dropout, pad_idx):super().__init__()self.embedding = nn.Embedding(vocab_size, embedding_dim, padding_idx=pad_idx)self.rnn = nn.LSTM(embedding_dim, hidden_dim, num_layers=n_layers, bidirectional=bidirectional, dropout=dropout)self.fc = nn.Linear(hidden_dim*2, output_dim)self.dropout = nn.Dropout(dropout)def forward(self, text):embedded = self.dropout(self.embedding(text)) #[sent len, batch size, emb dim]output, (hidden, cell) = self.rnn(embedded)#output = [sent len, batch size, hid dim * num directions]#hidden = [num layers * num directions, batch size, hid dim]#cell = [num layers * num directions, batch size, hid dim]#concat the final forward (hidden[-2,:,:]) and backward (hidden[-1,:,:]) hidden layers#and apply dropouthidden = self.dropout(torch.cat((hidden[-2,:,:], hidden[-1,:,:]), dim=1)) # [batch size, hid dim * num directions]return self.fc(hidden.squeeze(0))
INPUT_DIM = len(TEXT.vocab)
EMBEDDING_DIM = 100
HIDDEN_DIM = 256
OUTPUT_DIM = 1
N_LAYERS = 2
BIDIRECTIONAL = True
DROPOUT = 0.5
PAD_IDX = TEXT.vocab.stoi[TEXT.pad_token]model = RNN(INPUT_DIM, EMBEDDING_DIM, HIDDEN_DIM, OUTPUT_DIM, N_LAYERS, BIDIRECTIONAL, DROPOUT, PAD_IDX)
print(f'The model has {count_parameters(model):,} trainable parameters')
model.embedding.weight.data.copy_(pretrained_embeddings)
UNK_IDX = TEXT.vocab.stoi[TEXT.unk_token]model.embedding.weight.data[UNK_IDX] = torch.zeros(EMBEDDING_DIM)
model.embedding.weight.data[PAD_IDX] = torch.zeros(EMBEDDING_DIM)print(model.embedding.weight.data)
结果：
The model has 4,810,857 trainable parameters
tensor([[ 0.0000, 0.0000, 0.0000, ..., 0.0000, 0.0000, 0.0000],[ 0.0000, 0.0000, 0.0000, ..., 0.0000, 0.0000, 0.0000],[-0.0382, -0.2449, 0.7281, ..., -0.1459, 0.8278, 0.2706],...,[-0.7244, -0.0186, 0.0996, ..., 0.0045, -1.0037, 0.6646],[-1.1243, 1.2040, -0.6489, ..., -0.7526, 0.5711, 1.0081],[ 0.0860, 0.1367, 0.0321, ..., -0.5542, -0.4557, -0.0382]],device='cuda:0')
（1）训练RNN模型
optimizer = optim.Adam(model.parameters())
model = model.to(device)
N_EPOCHS = 5
best_valid_loss = float('inf')
for epoch in range(N_EPOCHS):start_time = time.time()train_loss, train_acc = train(model, train_iterator, optimizer, criterion)valid_loss, valid_acc = evaluate(model, valid_iterator, criterion)end_time = time.time()epoch_mins, epoch_secs = epoch_time(start_time, end_time)if valid_loss < best_valid_loss:best_valid_loss = valid_losstorch.save(model.state_dict(), 'lstm-model.pt')print(f'Epoch: {epoch+1:02} | Epoch Time: {epoch_mins}m {epoch_secs}s')print(f'\tTrain Loss: {train_loss:.3f} | Train Acc: {train_acc*100:.2f}%')print(f'\t Val. Loss: {valid_loss:.3f} | Val. Acc: {valid_acc*100:.2f}%')
结果：
Epoch: 01 | Epoch Time: 1m 29sTrain Loss: 0.676 | Train Acc: 57.69%Val. Loss: 0.694 | Val. Acc: 53.40%
Epoch: 02 | Epoch Time: 1m 29sTrain Loss: 0.641 | Train Acc: 63.77%Val. Loss: 0.744 | Val. Acc: 49.22%
Epoch: 03 | Epoch Time: 1m 29sTrain Loss: 0.618 | Train Acc: 65.77%Val. Loss: 0.534 | Val. Acc: 73.72%
Epoch: 04 | Epoch Time: 1m 30sTrain Loss: 0.634 | Train Acc: 63.79%Val. Loss: 0.619 | Val. Acc: 66.85%
Epoch: 05 | Epoch Time: 1m 29sTrain Loss: 0.448 | Train Acc: 79.19%Val. Loss: 0.340 | Val. Acc: 86.63%
4. CNN模型
class CNN(nn.Module):def __init__(self, vocab_size, embedding_dim, n_filters, filter_sizes, output_dim, dropout, pad_idx):super().__init__()self.embedding = nn.Embedding(vocab_size, embedding_dim, padding_idx=pad_idx)self.convs = nn.ModuleList([nn.Conv2d(in_channels = 1, out_channels = n_filters, kernel_size = (fs, embedding_dim)) for fs in filter_sizes])self.fc = nn.Linear(len(filter_sizes) * n_filters, output_dim)self.dropout = nn.Dropout(dropout)def forward(self, text):text = text.permute(1, 0) # [batch size, sent len]embedded = self.embedding(text) # [batch size, sent len, emb dim]embedded = embedded.unsqueeze(1) # [batch size, 1, sent len, emb dim]conved = [F.relu(conv(embedded)).squeeze(3) for conv in self.convs]#conv_n = [batch size, n_filters, sent len - filter_sizes[n]]pooled = [F.max_pool1d(conv, conv.shape[2]).squeeze(2) for conv in conved]#pooled_n = [batch size, n_filters]cat = self.dropout(torch.cat(pooled, dim=1))#cat = [batch size, n_filters * len(filter_sizes)]return self.fc(cat)
INPUT_DIM = len(TEXT.vocab)
EMBEDDING_DIM = 100
N_FILTERS = 100
FILTER_SIZES = [3,4,5]
OUTPUT_DIM = 1
DROPOUT = 0.5
PAD_IDX = TEXT.vocab.stoi[TEXT.pad_token]model = CNN(INPUT_DIM, EMBEDDING_DIM, N_FILTERS, FILTER_SIZES, OUTPUT_DIM, DROPOUT, PAD_IDX)
model.embedding.weight.data.copy_(pretrained_embeddings)
UNK_IDX = TEXT.vocab.stoi[TEXT.unk_token]model.embedding.weight.data[UNK_IDX] = torch.zeros(EMBEDDING_DIM)
model.embedding.weight.data[PAD_IDX] = torch.zeros(EMBEDDING_DIM)
model = model.to(device)
optimizer = optim.Adam(model.parameters())
criterion = nn.BCEWithLogitsLoss()
criterion = criterion.to(device)N_EPOCHS = 5best_valid_loss = float('inf')for epoch in range(N_EPOCHS):start_time = time.time()train_loss, train_acc = train(model, train_iterator, optimizer, criterion)valid_loss, valid_acc = evaluate(model, valid_iterator, criterion)end_time = time.time()epoch_mins, epoch_secs = epoch_time(start_time, end_time)if valid_loss < best_valid_loss:best_valid_loss = valid_losstorch.save(model.state_dict(), 'CNN-model.pt')print(f'Epoch: {epoch+1:02} | Epoch Time: {epoch_mins}m {epoch_secs}s')print(f'\tTrain Loss: {train_loss:.3f} | Train Acc: {train_acc*100:.2f}%')print(f'\t Val. Loss: {valid_loss:.3f} | Val. Acc: {valid_acc*100:.2f}%')
结果：
Epoch: 01 | Epoch Time: 0m 11sTrain Loss: 0.645 | Train Acc: 62.12%Val. Loss: 0.485 | Val. Acc: 79.61%
Epoch: 02 | Epoch Time: 0m 11sTrain Loss: 0.423 | Train Acc: 80.59%Val. Loss: 0.360 | Val. Acc: 84.63%
Epoch: 03 | Epoch Time: 0m 11sTrain Loss: 0.302 | Train Acc: 87.33%Val. Loss: 0.320 | Val. Acc: 86.59%
Epoch: 04 | Epoch Time: 0m 11sTrain Loss: 0.222 | Train Acc: 91.20%Val. Loss: 0.306 | Val. Acc: 87.17%
Epoch: 05 | Epoch Time: 0m 11sTrain Loss: 0.161 | Train Acc: 93.99%Val. Loss: 0.325 | Val. Acc: 86.82%
model.load_state_dict(torch.load('CNN-model.pt'))
test_loss, test_acc = evaluate(model, test_iterator, criterion)
print(f'Test Loss: {test_loss:.3f} | Test Acc: {test_acc*100:.2f}%')
结果：
Test Loss: 0.336 | Test Acc: 85.66%
PyTorch入门-自然语言分类任务相关推荐
- pytorch 入门学习多分类问题-9
pytorch 入门学习多分类问题 运行结果 [1, 300] loss: 2.287[1, 600] loss: 2.137[1, 900] loss: 1.192 Accuracy on test ...
- pytorch 入门学习使用逻辑斯蒂做二分类-6
pytorch 入门学习使用逻辑斯蒂做二分类 使用pytorch实现逻辑斯蒂做二分类 import torch import torchvision import numpy as np import ...
- PyTorch入门（二）搭建MLP模型实现分类任务
本文是PyTorch入门的第二篇文章,后续将会持续更新,作为PyTorch系列文章. 本文将会介绍如何使用PyTorch来搭建简单的MLP(Multi-layer Perceptron,多层感 ...
- 基于Transformers入门自然语言处理！
前言 读者朋友们好,我是多多,许久未见,甚是想念. 我最近忙了1件自己感觉有意义的事情,特来分享.我将之前零零散散的Transformer博客.讲解文章进行了整理,形成了一个完整的教程叫做:learn ...
- 网易云课程：深度学习与PyTorch入门实战
网易云课程:深度学习与PyTorch入门实战 01 深度学习初见 1.1 深度学习框架简介 1.2 pytorch功能演示 2开发环境安装 3回归问题 3.1简单的回归问题(梯度下降算法) 3.3回归 ...
- 零基础入门自然语言处理的学习建议
在入门的阶段最适合做的事情: (1)阅读和学习自然语言处理(natural language processing,nlp)综述类文章和图书,对nlp有一个基本的认识,梳理nlp研究内容的演变,包括n ...
- 简单有趣的 NLP 教程：手把手教你用 PyTorch 辨别自然语言（附代码）
简单有趣的 NLP 教程:手把手教你用 PyTorch 辨别自然语言(附代码) 雷锋网(公众号:雷锋网)按:本文作者甄冉冉,原载于作者个人博客,雷锋网已获授权. 最近在学pyTorch的实际应用 ...
- 不是你无法入门自然语言处理（NLP），而是你没找到正确的打开
不是你无法入门自然语言处理(NLP),而是你没找到正确的打开 小狼 2017-05-24 9:16:08 数据挖掘 评论(0) 作者:Mr.Scofield 〇.序 之前一段时间,在结合深度学习 ...
- numpy pytorch 接口对应_拆书分享篇深度学习框架PyTorch入门与实践
<<深度学习框架PyTorch入门与实践>>读书笔记 <深度学习框架PyTorch入门与实践>读后感 小作者:马苗苗 读完<<深度学习框架PyTorc ...
最新文章
- sweetalert 1.0多次回调函数bug
- MEF实现设计上的“松耦合”（一）
- SqlZoo.net习题答案：Using the SELECT statement.【nobel】
- BI项目实施完成，不知道怎么进行测试？Smartbi手把手教你
- window7 右下角图标消失解决方法
- 豆瓣电影喜剧榜数据爬取
- KETTLE4个工作中有用的复杂实例--1、数据定时自动（自动抽取）同步作业
- 基于卡方分布的独立性检验
- 【心电信号】基于matlab心率检测【含Matlab源码 1993期】
- ElasticSearch倒排索引原理 数据的写入与查询过程
- nginx解决浏览器跨域问题_Nginx 反向代理解决浏览器跨域问题
- git如何在某个分支下创建分支
- 简单入门排序算法（直接插入排序，折半插入排序，希尔排序，冒泡排序，堆排序，归并排序）
- python-opencv创建空白图
- switch服务器帐户注销,堡垒之夜NS版Guest账号怎么更换_Guest账号退出更换方法介绍...
- Hieroglyph3(基于DirectX 11的开源渲染引擎) 框架分析
- win11抢先版体验
- TCP的分片和IP分片的区别
- 关于＜graphics.h＞的头文件
- JVM老生代增长过快问题排查