基于nao机器人实现语音对话（智能版本）

nao机器人实现语音对话

1、语音获取

nao耳麦有一个功能，它可以通过声音大小判断能力值，也就是声音越大能量越大。所以我们此次项目主要运用的就是nao 的这个功能，来展开实现的。下面是流程图。

功能流程图

我们可以看见上面的流程图，从录音开始到录音结束逻辑还是比较复杂的，而且还有一些我没画出来，这只是大概。
录音的代码

 def recorder(self):self.audio_recorder.stopMicrophonesRecording()time.sleep(0.2)energy = self.energy()print(energy['left'])self.audio_recorder.startMicrophonesRecording(record_path, "wav", 16000, (0,0,1,0))   print("record begin")wait = 0global flagflag = 0global flag_oneflag_one = 0while 1:energy = self.energy()time.sleep(0.1) if energy['left'] < 400 and wait <5:print("no body:",energy['left'],float(wait))wait +=0.1elif energy['left'] >600:print("have people:",energy['left'],wait)wait = 4.7continueelif wait >= 5:print("record over ")self.audio_recorder.stopMicrophonesRecording()breakelse:continueif round(wait,1) == 4.0:print("int wait:",int(wait))self.answer_nao.say("你还有什么要说的吗,没有我要休眠咯")self.audio_recorder.stopMicrophonesRecording()time.sleep(1)self.audio_recorder.startMicrophonesRecording(record_path,"wav",16000,(0,0,1,0))wait = 3flag = 1while 1:energy = self.energy()time.sleep(0.1) if energy['left'] < 400 and wait <5:print("no body:",energy['left'],float(wait))wait +=0.1elif energy['left'] >600:print("have people:",energy['left'],wait)wait = 4.7continueelif round(wait,1) == 4.8:self.answer_nao.say("慢走，期待下次与您相遇")self.audio_recorder.stopMicrophonesRecording()returnelif wait >= 5:print("record over ")self.audio_recorder.stopMicrophonesRecording()breakelse:continuemsg = listen()msg = str(msg)print(msg)time.sleep(1)if "没" in msg:self.answer_nao.say("拜拜")time.sleep(2)breakelif msg == "None":self.answer_nao.say("很高心跟您对话，期待再次与您相见")time.sleep(2)breakelif msg == "":self.answer_nao.say("期待再次和您相遇，再见")time.sleep(2)breakelse:flag_one = 2botMsg = turing.botInteraction(msg)test = str(botMsg)answerNao.say(test)time.sleep(0.5)audio.recorder()if flag == 1:break

nao一共有四个声道，下面的代码是获取声音能量值

获取能量值

def energy(self):energy = dict()       energy['left'] = self.audio_device.getLeftMicEnergy()energy['right'] = self.audio_device.getRightMicEnergy()energy['front'] = self.audio_device.getFrontMicEnergy()energy['rear'] = self.audio_device.getRearMicEnergy()return energy

上面代码就是通过声音能量值来判断是否有人说话，从而判断是否录音的。里面逻辑需要大家自己看看，我说不太清楚。。。

2、录音翻译成文本

得到录音需要用百度语音合成API将音频转换成文字。实现工程如下。

大家先去申请语音识别的API有了API KEY 和API 密码之后才行。

代码

from aip import AipSpeech
APP_ID = '21xxxxx'
API_KEY = 'O0gzDUHKkciBa60Vxxxxx'
SECRET_KEY = 'Psji0dC90D1OehYh63ZaQuc7xxxxxxx'
client = AipSpeech(APP_ID, API_KEY, SECRET_KEY)def listen():with open(record_path, 'rb') as fp:voices = fp.read()try:result = client.asr(voices, 'wav', 16000, {'dev_pid': 1537, })result_text = result["result"][0]result_text = result_text.replace(',','')result_text = result_text.replace('.','')return result_textexcept KeyError:print("faild")

3、将文本传给华为云

这里主要是将问题传给华为云知识库，看是否能找到匹配的问题，比如我们做的是农业相关的，我就在华为云知识库中添加农业相关的知识。

华为云机器人

代码

后续添加，代码在机器人里

点击进来我们就可以看见知识库了。可以自己添加。也可以添加技能，这个就要自己对华为云机器人熟悉了，我就不够多的阐述。

4、传给图灵机器人

如果华为云知识库没有的问题就可以传给图灵机器人人了，因为图灵机器人闲聊比华为云机器人好一些。

创建之后也可以添加问题，不过没有华为云的好。

代码

def __init__(self):self.turing_url = 'http://www.tuling123.com/openapi/api?'def botInteraction (self,text):url_data = dict(key = 'e7ea86036040426e8a9d123176bfe12f',info = text,userid = 'yjc',)self.request = Request(self.turing_url + urlencode(url_data))try:w_data = urlopen(self.request)except URLError:raise Exception("No internet connection available to transfer txt data")except:raise KeyError("Server wouldn't respond (invalid key or quota has been maxed out)")response_text = w_data.read().decode('utf-8')json_result = json.loads(response_text)return json_result['text']

然后就可以语音把传回来的就可以拿回来语音播报了。

5、源码

# -*- coding:utf-8 -*-
#!/usr/bin/env python
import argparse
from naoqi import ALProxy
import wave
import json
import sys
import os
import paho.mqtt.client as mqtt
import time
import requests
import re
import json
from time import sleep
import random
import json
import sys
import qi
import time
import tempfile
import requestsfrom scipy.io import wavfile
tts = audio = record = aup = None
record_path = '/home/nao/record.wav'
from aip import AipSpeech
from urllib2 import urlopen,Request
from urllib2 import URLError
from urllib import urlencode
reload(sys)
sys.setdefaultencoding('utf-8')
APP_ID = '21715692'
API_KEY = 'O0gzDUHKkciBa60VddBgzuO1'
SECRET_KEY = 'Psji0dC90D1OehYh63ZaQuc7UPA8soxb'
username = 'h_y8689'
user_demain_id = '0a37c79c8300f3840f9cc0137d392600'
project_name = 'cn-north-4'
project_domain_id = '0a37c81fbe00f38b2f0ac0135b8e3f93'
password = 'hjy123456789'
client = AipSpeech(APP_ID, API_KEY, SECRET_KEY)
sys.setdefaultencoding('utf-8')
global flag_two
flag_two = 0
sys.path.append(os.path.abspath(os.path.dirname(__file__) + '/' + '..'))
sys.path.append("..")
TASK_TOPIC = 'test'
client_id = time.strftime('%Y%m%d%H%M%S',time.localtime(time.time()))
client = mqtt.Client(client_id, transport='tcp')
client.connect("59.110.42.24", 1883, 60)
client.loop_start()def clicent_main(message: str):time_now = time.strftime('%Y-%m-%d %H-%M-%S', time.localtime(time.time()))payload = {"msg": "%s" % message, "data": "%s" % time_now}# publish(主题：Topic; 消息内容)client.publish(TASK_TOPIC, json.dumps(payload, ensure_ascii=False))print("Successful send message!")return Trueclass Audio:def __init__(self, audio_recorder, audio_device, answer_nao):self.audio_recorder = audio_recorderself.audio_device = audio_deviceself.answer_nao = answer_naoself.data_result = Nonedef recorder(self):self.audio_recorder.stopMicrophonesRecording()time.sleep(0.2)energy = self.energy()print(energy['left'])self.audio_recorder.startMicrophonesRecording(record_path, "wav", 16000, (0,0,1,0))   print("record begin")wait = 0global flagflag = 0global flag_oneflag_one = 0while 1:energy = self.energy()time.sleep(0.1) if energy['left'] < 400 and wait <5:print("no body:",energy['left'],float(wait))wait +=0.1elif energy['left'] >600:print("have people:",energy['left'],wait)wait = 4.8continueelif wait >= 5:print("record over ")self.audio_recorder.stopMicrophonesRecording()breakelse:continueif round(wait,1) == 4.0:print("int wait:",int(wait))self.answer_nao.say("你还有什么要说的吗,没有我要休眠咯")self.audio_recorder.stopMicrophonesRecording()time.sleep(1)self.audio_recorder.startMicrophonesRecording(record_path,"wav",16000,(0,0,1,0))wait = 2flag = 1while 1:energy = self.energy()time.sleep(0.1) if energy['left'] < 400 and wait <5:print("no body:",energy['left'],float(wait))wait +=0.1elif energy['left'] >600:print("have people:",energy['left'],wait)wait = 4.9continueelif wait >= 5:print("record over ")self.audio_recorder.stopMicrophonesRecording()breakelse:continueif round(wait,1)==4.6:self.answer_nao.say("期待与您再次相遇")self.audio_recorder.stopMicrophonesRecording()flag_two = 2returnmsg = listen()msg = str(msg)print(msg)time.sleep(1)if "没" in msg:self.answer_nao.say("拜拜")time.sleep(2)breakelif msg == "None":self.answer_nao.say("很高心跟您对话，期待再次与您相见")time.sleep(2)breakelif msg == "":self.answer_nao.say("期待再次和您相遇，再见")time.sleep(2)breakelse:flag_one = 2botMsg = turing.botInteraction(msg)test = str(botMsg)answerNao.say(test)time.sleep(0.5)audio.recorder()if flag == 1:breakdef energy(self):energy = dict()       energy['left'] = self.audio_device.getLeftMicEnergy()energy['right'] = self.audio_device.getRightMicEnergy()energy['front'] = self.audio_device.getFrontMicEnergy()energy['rear'] = self.audio_device.getRearMicEnergy()return energydef answer(self, answer_data):self.answer_nao.setLanguage("Chinese")self.answer_nao.say(answer_data)def main(session):audioRecorder = session.service('ALAudioRecorder') audioDevice = session.service('ALAudioDevice')answerNao = session.service("ALTextToSpeech") audio = Audio(audioRecorder, audioDevice, answerNao)audio.recorder()try:passexcept Exception, errorMsg:print str(errorMsg)exit()class TuringChatMode(object):def __init__(self):self.turing_url = 'http://www.tuling123.com/openapi/api?'def botInteraction (self,text):url_data = dict(key = 'e7ea86036040426e8a9d123176bfe12f',info = text,userid = 'yjc',)self.request = Request(self.turing_url + urlencode(url_data))try:w_data = urlopen(self.request)except URLError:raise Exception("No internet connection available to transfer txt data")except:raise KeyError("Server wouldn't respond (invalid key or quota has been maxed out)")response_text = w_data.read().decode('utf-8')json_result = json.loads(response_text)return json_result['text']def main(robot_IP, robot_PORT=9559):global tts, audio, record, aup tts = ALProxy("ALTextToSpeech", robot_IP, robot_PORT)record = ALProxy("ALAudioRecorder", robot_IP, robot_PORT)aup = ALProxy("ALAudioPlayer", robot_IP, robot_PORT)print 'start recording...'record.startMicrophonesRecording(record_path, 'wav', 16000, (0,0,1,0))time.sleep(6)record.stopMicrophonesRecording()print 'record over'  def huawei(msg):url1 = 'https://iam.cn-north-4.myhuaweicloud.com/v3/auth/tokens'header ={'Content-Type': 'application/json;charset=utf8' }data = { "auth": { "identity": { "methods": [ "password" ], "password": { "user": { "name": "h_y8689", "password": "hjy123456789","domain": { "name": "h_y8689" } } } }, "scope": { "project": { "name": "cn-north-4" } } } }global aa = 0res1 = requests.post(url1,data=json.dumps(data),headers  =header)res1 = res1.headers['X-Subject-Token']#print("token:",res1[0:10])# url2 = 'https://cbs-ext.cn-north-4.myhuaweicloud.com/v1/0a37c81fbe00f38b2f0ac0135b8e3f93/qabots/5c71f659-3bc3-4f4b-8b1c-4125fcff7233/suggestions'Request_Header = {'Content-Type': 'application/json','X-Auth-Token' :res1}url2 = 'https://cbs-ext.cn-north-4.myhuaweicloud.com/v1/0a37c81fbe00f38b2f0ac0135b8e3f93/qabots/5c71f659-3bc3-4f4b-8b1c-4125fcff7233/sessions'res_2 = requests.post(url2, headers  = Request_Header)#print(res_2.text)res_2 = json.loads(res_2.text)def ques(que,res_2):url4 = 'https://cbs-ext.cn-north-4.myhuaweicloud.com/v1/0a37c81fbe00f38b2f0ac0135b8e3f93/qabots/5c71f659-3bc3-4f4b-8b1c-4125fcff7233/sessions/{}'.format(res_2['session_id'])body = {'question' : que,'top' : '1','tag_ids' : 'nao','domain_ids' : 'nao','chat_enable': 'true'}res_4 = requests.post(url4, data=json.dumps(body), headers  = Request_Header)res_4 = json.loads(res_4.text)return res_4que = msgres_4 = ques(que, res_2)print(res_4)if(res_4['reply_type'] == 0):#print(float(res_4['qabot_answers']['answers'][0]['score']))if(float(res_4['qabot_answers']['answers'][0]['score']) < 0.8):a = 1print(1)url5 = 'https://cbs-ext.cn-north-4.myhuaweicloud.com/v1/0a37c81fbe00f38b2f0ac0135b8e3f93/qabots/5c71f659-3bc3-4f4b-8b1c-4125fcff7233/sessions/{}'.format(res_2['session_id'])res_5 = requests.delete(url5, headers  = Request_Header)return else:print("2")answerNao.say(res_4['qabot_answers']['answers'][0]['answer'])url5 = 'https://cbs-ext.cn-north-4.myhuaweicloud.com/v1/0a37c81fbe00f38b2f0ac0135b8e3f93/qabots/5c71f659-3bc3-4f4b-8b1c-4125fcff7233/sessions/{}'.format(res_2['session_id'])res_5 = requests.delete(url5, headers  = Request_Header)returnelse:print(3)a = 1#answerNao.say(res_4['chat_answers']['answer'])url5 = 'https://cbs-ext.cn-north-4.myhuaweicloud.com/v1/0a37c81fbe00f38b2f0ac0135b8e3f93/qabots/5c71f659-3bc3-4f4b-8b1c-4125fcff7233/sessions/{}'.format(res_2['session_id'])res_5 = requests.delete(url5, headers  = Request_Header)returndef listen():with open(record_path, 'rb') as fp:voices = fp.read()try:result = client.asr(voices, 'wav', 16000, {'dev_pid': 1537, })result_text = result["result"][0]result_text = result_text.replace(',','')result_text = result_text.replace('.','')return result_textexcept KeyError:print("faild")if __name__ == "__main__":parser = argparse.ArgumentParser()parser.add_argument("--ip", type=str, default="192.168.1.89", help="Robot ip address")parser.add_argument("--port", type=int, default=9559, help="Robot port number")args = parser.parse_args()session = qi.Session()try:session.connect("tcp://" + args.ip + ":" + str(args.port))except RuntimeError:print("Can't connect to Naoqi at ip "" + args.ip + "" on port " + str(args.port) +"Please check your script arguments. Run with -h option for help.")sys.exit(1)turing = TuringChatMode()audioRecorder = session.service('ALAudioRecorder') audioDevice = session.service('ALAudioDevice')answerNao = session.service("ALTextToSpeech") audio = Audio(audioRecorder, audioDevice, answerNao)answerNao.setLanguage("Chinese") print("enter xunhuan")while 1:energy = audio.energy()print(energy['left'])if energy['left']>2000:answerNao.say("你好，很高兴认识你")time.sleep(0.5)while 1:audio.recorder()msg = listen()msg = str(msg)if (len(msg) <= 1):breakif "拜" in msg:answerNao.say("期待下次相遇")time.sleep(1)breakif "再见" in msg:answerNao.say("期待下次相遇")time.sleep(1)breakif “开灯”in msg：clicent_main("打开")if “开灯”in msg：clicent_main("关灯")if flag == 1:breakif flag_two == 2:breakhuawei(msg)print(a)if (a != 1):continuebotMsg = turing.botInteraction(msg)test = str(botMsg)answerNao.say(test)time.sleep(0.5)

基于nao机器人实现语音对话（智能版本）相关推荐

与图灵机器人进行语音对话（Java）(百度语音识别/合成)
使用Java编写一个与图灵机器人进行对话的工具. 但图灵机器人只支持文本对话,并不支持语音交互,所以本程序运用了第三方库百度语音识别/合成,来实现语音交互. 我们还可以将下载好的音乐放入指定文件夹,进 ...
基于单片机的无线语音遥控智能车设计
文末下载完整资料 1 无线语音控制智能车设计概述 1.1 设计目的随着电子工业的发展,计算机技术随着集成度的提高,和大规模集成电路的发展.计算机的一个重要分支--嵌入式计算机也在向高度智能化的 ...
一个基于nao机器人平台的高校迎新智能语音对话机器人
一.利用nao录制语音 nao录制音频保存在本地 # -*- coding: UTF-8 -*- #利用的naoqi中的ALProxy包import argparse from naoqi impor ...
有没有python与机械结合的工作-Python3从零开始搭建一个语音对话机器人的实现...
01-初心缘由最近在研究语音识别方向,看了很多的语音识别的资料和文章,了解了一下语音识别的前世今生,其中包含了很多算法的演变,目前来说最流行的语音识别算法主要是依赖于深度学习的神经网络算法,其中RN ...
python发音机器人_Python3从零开始搭建一个语音对话机器人
目录 01-初心缘由最近在研究语音识别方向,看了很多的语音识别的资料和文章,了解了一下语音识别的前世今生,其中包含了很多算法的演变,目前来说最流行的语音识别算法主要是依赖于深度学习的神经网络算法,其 ...
从零开始搭建一个语音对话机器人
点击上方[全栈开发者社区]→右上角[...]→[设为星标⭐] ♪ 点击上方绿标收听从零开始搭建一个语音对话机器人从零开始搭建一个语音对话机器人目录 01-初心缘由 01-准备工作 02-语音机器 ...
espeak 中文语音包_从零开始搭建一个语音对话机器人
点击上方[全栈开发者社区]→右上角[...]→[设为星标⭐] ♪ 点击上方绿标收听从零开始搭建一个语音对话机器人从零开始搭建一个语音对话机器人目录 01-初心缘由 01-准备工作 02-语音机器 ...
Nao机器人运行B-human代码之开始篇 1
申明:既然在网上有介绍安装B-human软件的,我就不写了,在此转载.原文地址:http://www.cnblogs.com/followyourheart/archive/2012/03/12/B- ...
STM32单片机语音声控智能台灯可调光冷暖光人检测锂电池供电太阳能和USB充电
实践制作DIY- GC0022-语音声控智能台灯一.功能说明: 基于STM32单片机设计-语音声控智能台灯功能介绍: 硬件:STM32F103C最小系统+语音识别模块+18650锂电池+太阳能充电 ...
51单片机语音声控智能台灯可调光冷暖光人检测锂电池供电太阳能和USB充电
实践制作DIY- GC0067-语音声控智能台灯一.功能说明: 基于51单片机设计-语音声控智能台灯二.功能介绍: 硬件:STC89C52单片机+语音识别模块+18650锂电池+太阳能充电板+LC ...