Android 语音遥控器的整体分析

今天蓝牙遥控器的导入终于完成了，分别梳理和记录一下部分语音和蓝牙相关的知识，首先是主机端上层语音部分：

一、应用层使用MediaRecorder的过程（应用层）

1.创建一个MediaRecorder

 mRecorder = new MediaRecorder();

2.设置录音来源

 mRecorder.setAudioSource(MediaRecorder.AudioSource.MIC);

3.设置录音比特率

 mRemainingTimeCalculator.setBitRate(SoundRecorder.BITRATE_3GPP);

4.设置采样率

mRecorder.setAudioSamplingRate(highQuality ? 44100 : 22050);

5.设置音频输出格式

  mRecorder.setOutputFormat(outputfileformat);

6.设置音频编码格式

 mRecorder.setAudioEncoder(MediaRecorder.AudioEncoder.AAC);

7.设置音频输出文件路径

mRecorder.setOutputFile(path);

8.设置异常监听

 mRecorder.setOnErrorListener(this);

9.调用prepare做录音准备工作

mRecorder.prepare();

10.正式开始录音

 mRecorder.start();

11.开始录音后可以调用WakeLock，方式屏幕关闭

mWakeLock.acquire();

12.可以通过广播改变UI状态

sendStateBroadcast();

二、应用层使用MediaRecorder的分析（框架层）

1.mRecorder.setOutputFormat(outputfileformat);中的outputfileformat是MediaRecorder.OutputFormat类型，新的音频格式可以在这里追加

    /*** Defines the output format. These constants are used with* {@link MediaRecorder#setOutputFormat(int)}.*/public final class OutputFormat {/* Do not change these values without updating their counterparts* in include/media/mediarecorder.h!*/private OutputFormat() {}public static final int DEFAULT = 0;/** 3GPP media file format*/public static final int THREE_GPP = 1;/** MPEG4 media file format*/public static final int MPEG_4 = 2;/** The following formats are audio only .aac or .amr formats *//*** AMR NB file format* @deprecated  Deprecated in favor of MediaRecorder.OutputFormat.AMR_NB*/public static final int RAW_AMR = 3;/** AMR NB file format */public static final int AMR_NB = 3;/** AMR WB file format */public static final int AMR_WB = 4;/** @hide AAC ADIF file format */public static final int AAC_ADIF = 5;/** AAC ADTS file format */public static final int AAC_ADTS = 6;/** @hide Stream over a socket, limited to a single stream */public static final int OUTPUT_FORMAT_RTP_AVP = 7;/** @hide H.264/AAC data encapsulated in MPEG2/TS */public static final int OUTPUT_FORMAT_MPEG2TS = 8;};

2. mRecorder.setAudioSamplingRate(highQuality ? 44100 : 22050);是设置音频的采样频率在prepare（）之前调用，因为在prepare中会对采样频率的可行性做检测。采样频率和音频的编码格式有关，比如AAC 编码的介于8k到96kHZ，AMRNB编码的为8kHZ，AMRWB为16kHZ，根据实际需要来设置。

   /*** Sets the audio sampling rate for recording. Call this method before prepare().* Prepare() may perform additional checks on the parameter to make sure whether* the specified audio sampling rate is applicable. The sampling rate really depends* on the format for the audio recording, as well as the capabilities of the platform.* For instance, the sampling rate supported by AAC audio coding standard ranges* from 8 to 96 kHz, the sampling rate supported by AMRNB is 8kHz, and the sampling* rate supported by AMRWB is 16kHz. Please consult with the related audio coding* standard for the supported audio sampling rate.** @param samplingRate the sampling rate for audio in samples per second.*/public void setAudioSamplingRate(int samplingRate) {if (samplingRate <= 0) {throw new IllegalArgumentException("Audio sampling rate is not positive");}setParameter("audio-param-sampling-rate=" + samplingRate);}

3. mRecorder.setAudioEncoder(MediaRecorder.AudioEncoder.AAC);就是设置前面设置采样频率时提到的编码格式了。这里编码格式定义在MediaRecorder.AudioEncoder这个内部类中，AudioDecoder也是定义在这里

   /*** Defines the audio encoding. These constants are used with* {@link MediaRecorder#setAudioEncoder(int)}.*/public final class AudioEncoder {/* Do not change these values without updating their counterparts* in include/media/mediarecorder.h!*/private AudioEncoder() {}public static final int DEFAULT = 0;/** AMR (Narrowband) audio codec */public static final int AMR_NB = 1;/** AMR (Wideband) audio codec */public static final int AMR_WB = 2;/** AAC Low Complexity (AAC-LC) audio codec */public static final int AAC = 3;/** High Efficiency AAC (HE-AAC) audio codec */public static final int HE_AAC = 4;/** Enhanced Low Delay AAC (AAC-ELD) audio codec */public static final int AAC_ELD = 5;}/*** Defines the video encoding. These constants are used with* {@link MediaRecorder#setVideoEncoder(int)}.*/public final class VideoEncoder {/* Do not change these values without updating their counterparts* in include/media/mediarecorder.h!*/private VideoEncoder() {}public static final int DEFAULT = 0;public static final int H263 = 1;public static final int H264 = 2;public static final int MPEG_4_SP = 3;}

4. mRecorder.prepare();中又做了什么？这里是调用jni函数去实现了。前面说过可以自定义编码格式，

    /*** Prepares the recorder to begin capturing and encoding data. This method* must be called after setting up the desired audio and video sources,* encoders, file format, etc., but before start().** @throws IllegalStateException if it is called after* start() or before setOutputFormat().* @throws IOException if prepare fails otherwise.*/public void prepare() throws IllegalStateException, IOException{_prepare();}

jni部分的实现是在\frameworks\av\media\libmedia中

status_t MediaRecorder::prepare()
{ALOGV("prepare");if (mMediaRecorder == NULL) {ALOGE("media recorder is not initialized yet");return INVALID_OPERATION;}if (!(mCurrentState & MEDIA_RECORDER_DATASOURCE_CONFIGURED)) {ALOGE("prepare called in an invalid state: %d", mCurrentState);return INVALID_OPERATION;}if (mIsAudioSourceSet != mIsAudioEncoderSet) {if (mIsAudioSourceSet) {ALOGE("audio source is set, but audio encoder is not set");} else {  // must not happen, since setAudioEncoder checks this alreadyALOGE("audio encoder is set, but audio source is not set");}return INVALID_OPERATION;}if (mIsVideoSourceSet != mIsVideoEncoderSet) {if (mIsVideoSourceSet) {ALOGE("video source is set, but video encoder is not set");} else {  // must not happen, since setVideoEncoder checks this alreadyALOGE("video encoder is set, but video source is not set");}return INVALID_OPERATION;}status_t ret = mMediaRecorder->prepare();if (OK != ret) {ALOGE("prepare failed: %d", ret);mCurrentState = MEDIA_RECORDER_ERROR;return ret;}mCurrentState = MEDIA_RECORDER_PREPARED;return ret;
}

主要是调用mMediaRecorder去实现
这个mMediaRecorder是个Mediarecorder的成员变量，是个强指针（注意，不是智能指针）

sp<IMediaRecorder>          mMediaRecorder;

5.现在要看这个IMediaRecorder类型的 mMediaRecorder->prepare()调用到了哪里，我们需要弄清楚MediaRecorder这一部分的类图：

显然是调用到了MediaRecorderClient.prepare()中

status_t MediaRecorderClient::prepare()
{ALOGV("prepare");Mutex::Autolock lock(mLock);if (mRecorder == NULL) {ALOGE("recorder is not initialized");return NO_INIT;}return mRecorder->prepare();
}

这里的mRecorder是一个MediaRecorderBase类型的成员，整个Android系统中目前还只有一个派生类StagefightRecorder。

所以根据继承关系知道最终是调用到了StagefightRecorder.prepare()和StagefrightRecorder.start()中，这里就正式开始录音咯！！可以看到前面设置的音频类型在这里会用到

status_t StagefrightRecorder::start() {// Get UID here for permission checkingmClientUid = IPCThreadState::self()->getCallingUid();if (mWriter != NULL) {ALOGE("File writer is not avaialble");return UNKNOWN_ERROR;}status_t status = OK;switch (mOutputFormat) {case OUTPUT_FORMAT_DEFAULT:case OUTPUT_FORMAT_THREE_GPP:case OUTPUT_FORMAT_MPEG_4:status = startMPEG4Recording();break;case OUTPUT_FORMAT_AMR_NB:case OUTPUT_FORMAT_AMR_WB:status = startAMRRecording();break;case OUTPUT_FORMAT_AAC_ADIF:case OUTPUT_FORMAT_AAC_ADTS:status = startAACRecording();break;case OUTPUT_FORMAT_RTP_AVP:status = startRTPRecording();break;case OUTPUT_FORMAT_MPEG2TS:status = startMPEG2TSRecording();break;default:ALOGE("Unsupported output file format: %d", mOutputFormat);status = UNKNOWN_ERROR;break;}if ((status == OK) && (!mStarted)) {mStarted = true;uint32_t params = IMediaPlayerService::kBatteryDataCodecStarted;if (mAudioSource != AUDIO_SOURCE_CNT) {params |= IMediaPlayerService::kBatteryDataTrackAudio;}if (mVideoSource != VIDEO_SOURCE_LIST_END) {params |= IMediaPlayerService::kBatteryDataTrackVideo;}addBatteryData(params);}return status;
}

现在分析下startAMRRecording（）;的具体实现

status_t StagefrightRecorder::startAMRRecording() {CHECK(mOutputFormat == OUTPUT_FORMAT_AMR_NB ||mOutputFormat == OUTPUT_FORMAT_AMR_WB);if (mOutputFormat == OUTPUT_FORMAT_AMR_NB) {if (mAudioEncoder != AUDIO_ENCODER_DEFAULT &&mAudioEncoder != AUDIO_ENCODER_AMR_NB) {ALOGE("Invalid encoder %d used for AMRNB recording",mAudioEncoder);return BAD_VALUE;}} else {  // mOutputFormat must be OUTPUT_FORMAT_AMR_WBif (mAudioEncoder != AUDIO_ENCODER_AMR_WB) {ALOGE("Invlaid encoder %d used for AMRWB recording",mAudioEncoder);return BAD_VALUE;}}mWriter = new AMRWriter(mOutputFd);status_t status = startRawAudioRecording();if (status != OK) {mWriter.clear();mWriter = NULL;}return status;
}

正在干活的是这里：

status_t StagefrightRecorder::startRawAudioRecording() {if (mAudioSource >= AUDIO_SOURCE_CNT) {ALOGE("Invalid audio source: %d", mAudioSource);return BAD_VALUE;}status_t status = BAD_VALUE;if (OK != (status = checkAudioEncoderCapabilities())) {return status;}sp<MediaSource> audioEncoder = createAudioSource();if (audioEncoder == NULL) {return UNKNOWN_ERROR;}CHECK(mWriter != 0);mWriter->addSource(audioEncoder);if (mMaxFileDurationUs != 0) {mWriter->setMaxFileDuration(mMaxFileDurationUs);}if (mMaxFileSizeBytes != 0) {mWriter->setMaxFileSize(mMaxFileSizeBytes);}mWriter->setListener(mListener);mWriter->start();return OK;
}

6.startRawAudioRecording（)

(1)首先检测波特率采样率和通道数是否符合要求

status_t StagefrightRecorder::checkAudioEncoderCapabilities() {clipAudioBitRate();clipAudioSampleRate();clipNumberOfAudioChannels();return OK;
}

（2）然后设置一个MediaWriter的对象，真正录音的对象就是这个mWriter

可以分析其中一个派生类的具体实现来看怎么录音的。
正在的录音是在这里：

status_t AMRWriter::threadFunc() {mEstimatedDurationUs = 0;mEstimatedSizeBytes = 0;bool stoppedPrematurely = true;int64_t previousPausedDurationUs = 0;int64_t maxTimestampUs = 0;status_t err = OK;prctl(PR_SET_NAME, (unsigned long)"AMRWriter", 0, 0, 0);while (!mDone) {MediaBuffer *buffer;err = mSource->read(&buffer);if (err != OK) {break;}if (mPaused) {buffer->release();buffer = NULL;continue;}mEstimatedSizeBytes += buffer->range_length();if (exceedsFileSizeLimit()) {buffer->release();buffer = NULL;notify(MEDIA_RECORDER_EVENT_INFO, MEDIA_RECORDER_INFO_MAX_FILESIZE_REACHED, 0);break;}int64_t timestampUs;CHECK(buffer->meta_data()->findInt64(kKeyTime, ×tampUs));if (timestampUs > mEstimatedDurationUs) {mEstimatedDurationUs = timestampUs;}if (mResumed) {previousPausedDurationUs += (timestampUs - maxTimestampUs - 20000);mResumed = false;}timestampUs -= previousPausedDurationUs;ALOGV("time stamp: %lld, previous paused duration: %lld",timestampUs, previousPausedDurationUs);if (timestampUs > maxTimestampUs) {maxTimestampUs = timestampUs;}if (exceedsFileDurationLimit()) {buffer->release();buffer = NULL;notify(MEDIA_RECORDER_EVENT_INFO, MEDIA_RECORDER_INFO_MAX_DURATION_REACHED, 0);break;}ssize_t n = write(mFd,(const uint8_t *)buffer->data() + buffer->range_offset(),buffer->range_length());if (n < (ssize_t)buffer->range_length()) {buffer->release();buffer = NULL;err = ERROR_IO;break;}if (err != OK) {break;}if (stoppedPrematurely) {stoppedPrematurely = false;}buffer->release();buffer = NULL;}if ((err == OK || err == ERROR_END_OF_STREAM) && stoppedPrematurely) {err = ERROR_MALFORMED;}close(mFd);mFd = -1;mReachedEOS = true;if (err == ERROR_END_OF_STREAM) {return OK;}return err;
}

我们关注的是两个点：

1.MediaBuffer *buffer;音频最后都是来源于这里：err = mSource->read(&buffer);。分析音频格式，可以从这个buffer入手。

2. mFd = open(filename, O_CREAT | O_LARGEFILE | O_TRUNC | O_RDWR, S_IRUSR | S_IWUSR);

最后音频内容是保存在这个里面了。

后面会针对这个buffer来分析Android 的音频底层实现机制并添加自己的音频编解码。

Android 语音遥控器的整体分析相关推荐

Android 语音遥控器的整体分析-HAL层的AudioFlinger
上篇说到语音部分最后会通过AudioFlinger来操作HAL层. 一.首先我们看下硬件接口层的接口(奇怪为什么只有Audio的hardwareinterface): (1)hardware\libh ...
Android 语音遥控器的整体分析-主机端语音解码的添加
前面几篇大致介绍了HAL层的实现方式.这里要介绍下如何在Android主机端的HAL层语音解码的添加. 一.首先需要了解libhardware.so(\libhardware\hardware.c) ...
Android 语音遥控器的整体分析-底层实现机制分析
前面我们知道Writer类中开了一个线程在进行实际的录音. Writer.addSource()中,有这样一段代码: const char *kHeader = isWide ? "#!AM ...
智能会议系统（34）---Android语音通话实现方案及相关技术介绍
Android语音通话实现方案及相关技术介绍 Android语音通话实现方案及相关技术介绍语音通话 Step1语音采集和输出 Step2编解码方式 Step3网络传输 Step4去噪声消回音语音通 ...
离线语音遥控器控制红外设备
离线语音遥控器开发板可以通过语音控制红外设备,如图所示 B站链接根据实际产品应用编写SU03T语音模块的命令词和回复语(此处以康佳加湿器为例),注意命令词编写规则. 设置SU03T语音模块收到命令词 ...
2022-2028全球与中国语音遥控器市场现状及未来发展趋势
2021年全球语音遥控器市场销售额达到了亿美元,预计2028年将达到亿美元,年复合增长率(CAGR)为 %(2022-2028).地区层面来看,中国市场在过去几年变化较快,2021年市场规模为百 ...
使用ZEGO SDK零基础搭建Android语音聊天应用
2021年初,Clubhouse引爆了在线语音聊天应用,字节跳动.小米.快手.映客等国内泛娱乐社交厂商已纷纷跟进,那么搭建一个类Clubhouse的语音聊天软件难吗? 其实Clubhouse的技术原理 ...
苹果曝光新一代Apple TV遥控器，语音遥控器未来可期
外媒 9to5Mac 的最新消息指出,苹果正在为 Apple TV 开发一款新的遥控器,这也印证了之前苹果打算更新 Siri 遥控器的传闻.这款新的 Apple TV 遥控器细节还不得而知,但消息源告 ...
vst开启语音服务器,VST语音遥控器体验记
体验信息 101245hu4k1a42krae4l4s.jpg (10.39 KB, 下载次数: 5) 2015-5-30 16:34 上传物品名称:VST智能语音遥控器适配物品Ⅰ 101502huu ...

Android 语音遥控器的整体分析

Android 语音遥控器的整体分析相关推荐

最新文章

热门文章