一.前言

在智能客服领域,需要具备识别用户输入拼音并将其转为中文,进而来搜索与之相关问题,但用户输入的拼音内容是一个长字符串,此时需要把字符串进行中文拼音音节拆分,用逗号’,'将每个音节分开,因此本篇文章主要介绍构建基于Trie的拼音音节模型,在下篇文章介绍拆分音节,再下下篇文章介绍拼音识别成中文

二.构建模型

2.1 Trie树简要介绍

Trie树形结构属于字典树,整个树的根节点是空的, 根节点的子节点可以有多个,每个子节点智能存放单个元素,用于我们拼音模型就是根节点的子结点有26个,分别由a-z组成,每个子节点又可以当作其下的根节点,Trie字典树节点的结构为:

class TrieNode:

def __init__(self):

self.value = None

self.children = {}

Trie树结构例图

2.2 构建拼音模型

首先我们需要中文拼音词库pinyin.txt,里面包含各种音节,将这些音节加入到trie树,变成拼音字典树,例如:

a

ai

aing

an

ang

ao

ba

bai

ban

bang

bao

bei

.

.

.

初始化字典树:

class Trie:

def __init__(self):

self.root = TrieNode() # 初始化节点

self.trie_path = Pinyin_Trie_Path # 生成模型的目录文件

self.pinyin_path = Pinyin_Path # 中文拼音库目录文件

插入trie节点:

def insert(self, key):

node = self.root

for char in key:

if char not in node.children:

child = TrieNode()

node.children[char] = child

node = child

else:

node = node.children[char]

node.value = key

遍历音节字符串获取单个字母i,如果trie树已存在该单个字母,则继续在该字母节点的子节点中添加,例如:目前有音节zhao,而trie树已有zh,则遍历zhao,添加a作为h的子节点,再添加o作为a的子节点.如果字母i不属于子节点中,则需要新建一个TrieNode节点,存放新的字母.

def build_trie(self):

trie = Trie()

for line in open(self.pinyin_path):

word = line.strip().lower()

trie.insert(word)

with open(self.trie_path, 'wb') as f:

pickle.dump(trie, f)

此段代码是构建拼音模型的代码,它逐行读取中文拼音词库pinyin.txt的拼音并转换成小写,并调用insert()方法插入Trie树中,最后导出拼音模型文件pinyin_Trie.model

def search(self, key):

node = self.root

matches = []

for char in key:

if char not in node.children:

break

node = node.children[char]

if node.value:

matches.append(node.value)

return matches

该代码主要功能是根据新输入的拼音去查找比对trie树的内容,比如输入zhuanye,比对trie后,会把[zhu,zhua,zhuan],[ye]返回

三.全部代码

Proj_path = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))

Pinyin_Trie_Path = os.path.join(Proj_path, "main/pinyin_Trie.model")

Pinyin_Path = os.path.join(Proj_path, "main/assets/dataset/pinyin.txt")

# 定义trie字典树节点

class TrieNode:

def __init__(self):

self.value = None

self.children = {}

class Trie:

def __init__(self):

self.root = TrieNode()

self.trie_path = Pinyin_Trie_Path

self.pinyin_path = Pinyin_Path

# 添加树节点Pinyin_Path

def insert(self, key):

node = self.root

for char in key:

if char not in node.children:

child = TrieNode()

node.children[char] = child

node = child

else:

node = node.children[char]

node.value = key

# 查找节点

def search(self, key):

node = self.root

matches = []

for char in key:

if char not in node.children:

break

node = node.children[char]

if node.value:

matches.append(node.value)

return matches

def build_trie(self):

trie = Trie()

for line in open(self.pinyin_path):

word = line.strip().lower()

trie.insert(word)

with open(self.trie_path, 'wb') as f:

pickle.dump(trie, f)

有问题的,欢迎评论一起讨论!