obs源码分析【五】：音频采集线程

在第三篇介绍了视频的线程，音频的线程代码也是在那一块儿：

if (!ResetAudio())throw "Failed to initialize audio";

音频线程的创建是在audio_output_open函数中

int audio_output_open(audio_t **audio, struct audio_output_info *info)
{。。。if (pthread_create(&out->thread, NULL, audio_thread, out) != 0)goto fail;。。。
}

obs在windows上采集音频用的是wasapi, 主要是在win-wasapi这个工程里面，在界面初始化时需要加载各种dll, 这里面就有个wasapi.dll，音频采集线程创建的代码如下

void WASAPISource::InitCapture()
{HRESULT res = client->GetService(__uuidof(IAudioCaptureClient),(void **)capture.Assign());if (FAILED(res))throw HRError("Failed to create capture context", res);res = client->SetEventHandle(receiveSignal);if (FAILED(res))throw HRError("Failed to set event handle", res);//创建音频采集线程captureThread = CreateThread(nullptr, 0, WASAPISource::CaptureThread,this, 0, nullptr);if (!captureThread.Valid())throw "Failed to create capture thread";client->Start();active = true;blog(LOG_INFO, "WASAPI: Device '%s' [%s Hz] initialized",device_name.c_str(), device_sample.c_str());
}

可以去看看线程函数 CaptureThread的代码

while (WaitForCaptureSignal(2, sigs, dur)) {if (!source->ProcessCaptureData()) {reconnect = true;break;}}

在点击【开始录制】按钮后，会触发相应的windows事件，然后进入这个while循环，调用ProcessCaptureData去采集处理音频数据，主要就是调用下面这句代码：

res = capture->GetBuffer(&buffer, &frames, &flags, &pos, &ts);

然后执行到下面这句代码

obs_source_output_audio(source, &data);

再就是

void obs_source_output_audio(obs_source_t *source,const struct obs_source_audio *audio)
{。。。source_output_audio_data(source, &data);。。。

source_output_audio_data这个函数就很重要了，他需要处理音视频同步

static void source_output_audio_data(obs_source_t *source,const struct audio_data *data)
{size_t sample_rate = audio_output_get_sample_rate(obs->audio.audio);struct audio_data in = *data;uint64_t diff;uint64_t os_time = os_gettime_ns();int64_t sync_offset;bool using_direct_ts = false;bool push_back = false;/* detects 'directly' set timestamps as long as they're within* a certain threshold */if (uint64_diff(in.timestamp, os_time) < MAX_TS_VAR) {source->timing_adjust = 0;source->timing_set = true;using_direct_ts = true;}if (!source->timing_set) {reset_audio_timing(source, in.timestamp, os_time);} else if (source->next_audio_ts_min != 0) {diff = uint64_diff(source->next_audio_ts_min, in.timestamp);/* smooth audio if within threshold */if (diff > MAX_TS_VAR && !using_direct_ts)handle_ts_jump(source, source->next_audio_ts_min,in.timestamp, diff, os_time);else if (diff < TS_SMOOTHING_THRESHOLD) {if (source->async_unbuffered && source->async_decoupled)source->timing_adjust = os_time - in.timestamp;in.timestamp = source->next_audio_ts_min;}}source->last_audio_ts = in.timestamp;source->next_audio_ts_min =in.timestamp + conv_frames_to_time(sample_rate, in.frames);in.timestamp += source->timing_adjust;pthread_mutex_lock(&source->audio_buf_mutex);if (source->next_audio_sys_ts_min == in.timestamp) {push_back = true;} else if (source->next_audio_sys_ts_min) {diff = uint64_diff(source->next_audio_sys_ts_min, in.timestamp);if (diff < TS_SMOOTHING_THRESHOLD) {push_back = true;/* This typically only happens if used with async video when* audio/video start transitioning in to a timestamp jump.* Audio will typically have a timestamp jump, and then video* will have a timestamp jump.  If that case is encountered,* just clear the audio data in that small window and force a* resync.  This handles all cases rather than just looping. */} else if (diff > MAX_TS_VAR) {reset_audio_timing(source, data->timestamp, os_time);in.timestamp = data->timestamp + source->timing_adjust;}}sync_offset = source->sync_offset;in.timestamp += sync_offset;in.timestamp -= source->resample_offset;source->next_audio_sys_ts_min =source->next_audio_ts_min + source->timing_adjust;if (source->last_sync_offset != sync_offset) {if (source->last_sync_offset)push_back = false;source->last_sync_offset = sync_offset;}if (source->monitoring_type != OBS_MONITORING_TYPE_MONITOR_ONLY) {if (push_back && source->audio_ts)source_output_audio_push_back(source, &in);elsesource_output_audio_place(source, &in);}pthread_mutex_unlock(&source->audio_buf_mutex);source_signal_audio_data(source, data, source_muted(source, os_time));
}

关于音视频同步的内容，可以去看看ffplay的源码解析，有个阈值TS_SMOOTHING_THRESHOLD，需要把音频、视频的每一帧时间戳的差值TS_SMOOTHING_THRESHOLD做比较，然后做延时还是快进。

wasapi是com接口，在使用之前需要做初始化，如下所示：

void WASAPISource::Initialize()
{HRESULT res;res = CoCreateInstance(__uuidof(MMDeviceEnumerator), nullptr,CLSCTX_ALL, __uuidof(IMMDeviceEnumerator),(void **)enumerator.Assign());if (FAILED(res))throw HRError("Failed to create enumerator", res);if (!InitDevice())return;device_name = GetDeviceName(device);if (!notify) {notify = new WASAPINotify(this);enumerator->RegisterEndpointNotificationCallback(notify);}HRESULT resSample;IPropertyStore *store = nullptr;PWAVEFORMATEX deviceFormatProperties;PROPVARIANT prop;resSample = device->OpenPropertyStore(STGM_READ, &store);if (!FAILED(resSample)) {resSample =store->GetValue(PKEY_AudioEngine_DeviceFormat, &prop);if (!FAILED(resSample)) {if (prop.vt != VT_EMPTY && prop.blob.pBlobData) {deviceFormatProperties =(PWAVEFORMATEX)prop.blob.pBlobData;device_sample = std::to_string(deviceFormatProperties->nSamplesPerSec);}}store->Release();}InitClient();if (!isInputDevice)InitRender();InitCapture();
}

win-wasapi.cpp这个文件，大部分都是对系统音频设备的操作，初始化wasapi, 获取设备，获取音频buffer等。

obs源码分析【五】：音频采集线程相关推荐

obs源码分析【一】：main函数
目录 main函数在哪里 obs项目架构 main函数浅析 crash的处理 obs配置 obs log 网络请求libcurl 主界面 Qt适配高dpi屏幕总结最近对obs的代码感兴趣了,在 ...
【Java 并发编程】线程池机制 ( 线程池执行任务细节分析 | 线程池执行 execute 源码分析 | 先创建核心线程 | 再放入阻塞队列 | 最后创建非核心线程 )
文章目录一.线程池执行任务细节分析二.线程池执行 execute 源码分析一.线程池执行任务细节分析线程池执行细节分析 : 核心线程数 101010 , 最大小成熟 202020 , 非核心线 ...
MPTCP 源码分析(五) 接收端窗口值
简述: 在TCP协议中影响数据发送的三个因素分别为:发送端窗口值.接收端窗口值和拥塞窗口值. 本文主要分析MPTCP中各个子路径对接收端窗口值rcv_wnd的处理. 接收端窗口值的初始化根据< ...
【转】ABP源码分析五：ABP初始化全过程
ABP在初始化阶段做了哪些操作,前面的四篇文章大致描述了一下. 为个更清楚的描述其脉络,做了张流程图以辅助说明.其中每一步都涉及很多细节,难以在一张图中全部表现出来.每一步的细节(会涉及到较多接口,类 ...
Mybatis-Spring源码分析(五) MapperMethod和MappedStatement解析
前言基本上这就是Mybatis-Spring源码的最后一篇了,如果想起来什么再单开博客.比起来Spring源码,Mybatis的确实简单一些,本篇就说一下Mybatis中两个十分重要的类Mapper ...
Vue.js 源码分析(五) 基础篇方法 methods属性详解
methods中定义了Vue实例的方法,官网是这样介绍的: 例如:: <!DOCTYPE html> <html lang="en"> <head&g ...
openstack-nova源码分析(五）flavor extra_specs 扩展属性
Flavors extra-specs (flavors, os-flavor-extra-specs) Flavor 扩展属性设置, 扩展属性可以用来对虚拟机做一些额外的限制设置,具体的参数,将在后 ...
ceph bluestore源码分析：C++ 获取线程id
阅读ceph源码过程中需要明确当前操作是由哪个线程发出,此时需要根据线程id来确认线程名称 C++获取线程id是通过系统调用来直接获取函数描述头文件:<sys/syscall.h> 函 ...
tcp/ip 协议栈Linux源码分析五 IPv6分片报文重组分析一
做防火墙模块的时候遇到过IPv6分片报文处理的问题是,当时的问题是netfilter无法基于传输层的端口拦截IPv6分片报文,但是IPv4的分片报文可以.分析了内核源码得知是因为netfilter的连 ...

obs源码分析【五】：音频采集线程

obs源码分析【五】：音频采集线程相关推荐

最新文章

热门文章