使用sws_scale转换视频、使用swr_convert转换音频
2024-06-11 02:11:26
使用Sws转换视频、使用swr_convert转换音频
环境
➜ ~ ffmpeg -version
ffmpeg version N-105035-g617452ce2c Copyright (c) 2000-2021 the FFmpeg developers
built with Apple clang version 12.0.5 (clang-1205.0.22.11)
configuration: --enable-gpl --enable-filter=aresample --enable-bsf=aac_adtstoasc --enable-small --enable-dwt --enable-lsp --enable-mdct --enable-rdft --enable-fft --enable-static --enable-version3 --enable-nonfree --enable-encoder=libfdk_aac --enable-encoder=libx264 --enable-decoder=mp3 --enable-libx264 --enable-libfdk_aac --enable-libmp3lame --extra-cflags=-I/usr/local/include --extra-ldflags=-L/usr/local/lib
libavutil 57. 13.100 / 57. 13.100
libavcodec 59. 15.101 / 59. 15.101
libavformat 59. 10.100 / 59. 10.100
libavdevice 59. 1.100 / 59. 1.100
libavfilter 8. 21.100 / 8. 21.100
libswscale 6. 1.102 / 6. 1.102
libswresample 4. 0.100 / 4. 0.100
libpostproc 56. 0.100 / 56. 0.100
用法
音频
可使用swr_convert对音频数据转换频率及格式。
- 关键函数的定义
/*** 这个函数可以创建SwrContext，并且可以设置通用参数。* * 这个函数不依赖`swr_alloc`来事先分配SwrContext，另一方面如果有事先使用`swr_alloc`分配好的SwrContext* 可以通过这个函数来设置参数。** @param s 设置SwrContext如果没有则可以设置为null.* @param out_ch_layout 输出的ch_layout* @param out_sample_fmt 输出的采样格式.* @param out_sample_rate 输出的采样频率* @param in_ch_layout 输入的ch_layout* @param in_sample_fmt 输入的音频格式.* @param in_sample_rate 输入的音频采样率* @param log_offset logging level offset* @param log_ctx parent logging context, can be NULL** @see swr_init(), swr_free()* @return NULL on error, allocated context otherwise*/struct SwrContext *swr_alloc_set_opts(struct SwrContext *s,int64_t out_ch_layout, enum AVSampleFormat out_sample_fmt, int out_sample_rate,int64_t in_ch_layout, enum AVSampleFormat in_sample_fmt, int in_sample_rate,int log_offset, void *log_ctx);/*** 在设置完参数之后使用这个函数进行初始化SwrContext* * Initialize context after user parameters have been set.* @note The context must be configured using the AVOption API.** @see av_opt_set_int()* @see av_opt_set_dict()** @param[in,out] s Swr context to initialize* @return AVERROR error code in case of failure.*/
int swr_init(struct SwrContext *s);/** 转换音频.** in 和 in_count 可以设置为0用来冲内部剩余的缓存。* 如果输入的数据大于输出的空间，输入将被缓存在内部** @param s 初始化好的SwrContext* @param out 输出buffer, 如果是packed音频则之后第一个数组被用到。* @param out_count 输出数据每个通道的采样数。* @param in 输入buffer, 如果是packed音频则之后第一个数组被用到。* @param in_count 输入数据每个通道的采样数。** @return 每个通道的样本数，如果是负数则出错。*/int swr_convert(struct SwrContext *s, uint8_t **out, int out_count,const uint8_t **in , int in_count);
- 示例
- 初始化
// 1. 设置通用参数
this->audio_swr_context_ = swr_alloc_set_opts(nullptr,av_get_default_channel_layout(output_channels_),AV_SAMPLE_FMT_S16,audio_codec_context_->sample_rate,audio_codec_context_->channel_layout,audio_codec_context_->sample_fmt,audio_codec_context_->sample_rate,0,nullptr
);
if (this->audio_swr_context_ == nullptr) {throw std::runtime_error("audio_swr_context fail.");
}
// 2. 初始化
swr_init(this->audio_swr_context_);
- 转换
uint8_t **audio_dst_data = nullptr;
int linesize = 0;
// 为输出分配缓存
av_samples_alloc_array_and_samples(&audio_dst_data, &linesize, output_channels_,frame->nb_samples, AV_SAMPLE_FMT_S16, 1);
// 转换
int num_frames = swr_convert(audio_swr_context_,audio_dst_data,frame->nb_samples,const_cast<const uint8_t **>(frame->data),frame->nb_samples);int unpadded_line_size = num_frames * av_get_bytes_per_sample(AV_SAMPLE_FMT_S16) * output_channels_;
std::cout << vformat("Write audio frame %d,size=%d", this->audio_frame_counter_++, unpadded_line_size)<< std::endl;
output_audio_stream_.write(reinterpret_cast<const char *>(audio_dst_data[0]), unpadded_line_size);
// 释放缓存
if (&audio_dst_data[0]) {av_freep(&audio_dst_data[0]);
}
av_freep(&audio_dst_data);
视频
可以使用sws_scale进行视频转换
- 关键函数定义
/*** 分配并返回SwsContext。可以使用它去执行sws_scale来转换视频格式** @param srcW 原图的宽* @param srcH 原图的高* @param srcFormat 原图格式* @param dstW 输出图的宽* @param dstH 输出图的高* @param dstFormat 输出图的格式* @param flags 可以指定选用哪种算法。* @param param extra parameters to tune the used scaler* For SWS_BICUBIC param[0] and [1] tune the shape of the basis* function, param[0] tunes f(1) and param[1] f´(1)* For SWS_GAUSS param[0] tunes the exponent and thus cutoff* frequency* For SWS_LANCZOS param[0] tunes the width of the window function* @return 指向SwsContext的指针*/
struct SwsContext *sws_getContext(int srcW, int srcH, enum AVPixelFormat srcFormat,int dstW, int dstH, enum AVPixelFormat dstFormat,int flags, SwsFilter *srcFilter,SwsFilter *dstFilter, const double *param);/*** 缩放图像** @param c sws_getContext()创建的SwsContext* * @param srcSlice 输入数据，指向每个plane的指针数组* @param srcStride 输入数据，每个plane的宽度* @param srcSliceY 从哪个位置开始处理数据。* @param srcSliceH 输入数据的高* @param dst 输出数据，指向每个plane的指针数组* @param dstStride 输出数据，每个plane的宽度* @return 输出数据的高度。*/
int sws_scale(struct SwsContext *c, const uint8_t *const srcSlice[],
const int srcStride[], int srcSliceY, int srcSliceH,
uint8_t *const dst[], const int dstStride[]);
- 使用示例
- 创建SwsContext
// 创建SwsContext,把原视频尺寸缩小一半
this->video_sws_context_ = sws_getContext(
this->video_codec_context_->width,
this->video_codec_context_->height,
this->video_codec_context_->pix_fmt,
this->video_codec_context_->width / 2,
this->video_codec_context_->height / 2,
this->video_codec_context_->pix_fmt,
SWS_FAST_BILINEAR,
nullptr, nullptr, nullptr);
- 使用
sws_scale
int h = sws_scale(this->video_sws_context_,src_image_data_,src_image_data_line_size_,0,frame->height,dst_image_data_,dst_image_data_line_size_);output_video_stream_.write(reinterpret_cast<const char *>(this->dst_image_data_[0]), this->dst_image_data_buffer_size_);std::cout << vformat("sws_scale return %d",h) << std::endl;
- 分配图片缓存的方式
void Demuxing::AllocSrcImage() {this->src_image_data_buffer_size_ = av_image_alloc(this->src_image_data_,this->src_image_data_line_size_,this->video_codec_context_->width,this->video_codec_context_->height,this->video_codec_context_->pix_fmt,1);if (this->src_image_data_buffer_size_ < 0) {throw std::runtime_error(vformat("Alloc image error,message:%s", av_err2str(this->src_image_data_buffer_size_)));}
}
void Demuxing::AllocDstImage() {this->dst_image_data_buffer_size_= av_image_alloc(this->dst_image_data_,this->dst_image_data_line_size_,this->video_codec_context_->width /2,this->video_codec_context_->height /2,this->video_codec_context_->pix_fmt,1);if (this->dst_image_data_buffer_size_ < 0) {throw std::runtime_error(vformat("Alloc image error,message:%s", av_err2str(this->src_image_data_buffer_size_)));}
}
代码
//
// Created by blueberry on 2022/1/27.
//#include "Demuxing.h"#include "string_util.h"
#include <ios>Demuxing::Demuxing() = default;Demuxing::~Demuxing() {av_freep(this->src_image_data_);av_freep(this->dst_image_data_);av_free(this->av_packet_);av_free(this->av_frame_);avcodec_free_context(&this->video_codec_context_);avcodec_free_context(&this->audio_codec_context_);avformat_close_input(&format_context_);avformat_free_context(format_context_);this->output_video_stream_.close();this->output_audio_stream_.close();
};void Demuxing::Initialize(std::string input_file, std::string output_video, std::string output_audio) {this->input_file_ = std::move(input_file);this->output_audio_ = std::move(output_audio);this->output_video_ = std::move(output_video);format_context_ = avformat_alloc_context();avformat_open_input(&format_context_, this->input_file_.c_str(), nullptr, nullptr);avformat_find_stream_info(format_context_, nullptr);OpenCodecContext(AVMEDIA_TYPE_VIDEO,&this->video_codec_context_, &this->video_stream_index_);OpenCodecContext(AVMEDIA_TYPE_AUDIO,&this->audio_codec_context_, &this->audio_stream_index_);// 创建SwsContext,把原视频尺寸缩小一半this->video_sws_context_ = sws_getContext(this->video_codec_context_->width,this->video_codec_context_->height,this->video_codec_context_->pix_fmt,this->video_codec_context_->width / 2,this->video_codec_context_->height / 2,this->video_codec_context_->pix_fmt,SWS_FAST_BILINEAR,nullptr, nullptr, nullptr);if (this->video_codec_context_ == nullptr) {throw std::runtime_error("sws_getContext fail.");}// 1. 设置通用参数this->audio_swr_context_ = swr_alloc_set_opts(nullptr,av_get_default_channel_layout(output_channels_),AV_SAMPLE_FMT_S16,audio_codec_context_->sample_rate,audio_codec_context_->channel_layout,audio_codec_context_->sample_fmt,audio_codec_context_->sample_rate,0,nullptr);if (this->audio_swr_context_ == nullptr) {throw std::runtime_error("audio_swr_context fail.");}// 2. 初始化swr_init(this->audio_swr_context_);this->output_audio_stream_.open(this->output_audio_.c_str(), std::ios::binary | std::ios::out);this->output_video_stream_.open(this->output_video_.c_str(), std::ios::binary | std::ios::out);if (this->output_video_stream_.bad()) {throw std::runtime_error("Open output video file failed.");}if (this->output_audio_stream_.bad()) {throw std::runtime_error("Open output audio file failed.");}
}void Demuxing::Start() {av_dump_format(this->format_context_, 0, this->input_file_.c_str(), 0);this->av_packet_ = av_packet_alloc();this->av_frame_ = av_frame_alloc();AllocSrcImage();AllocDstImage();if (this->av_packet_ == nullptr) {throw std::runtime_error("alloc packet failure.");}if (this->av_frame_ == nullptr) {throw std::runtime_error("alloc avframe failure.");}while (av_read_frame(this->format_context_, this->av_packet_) >= 0) {int ret = 0;if (this->av_packet_->stream_index == this->audio_stream_index_) {// decode audio dataDecodePacket(this->audio_codec_context_);} else if (this->av_packet_->stream_index == this->video_stream_index_) {// decode video dataDecodePacket(this->video_codec_context_);}av_packet_unref(this->av_packet_);if (ret < 0) {break;}}
}int Demuxing::DecodePacket(AVCodecContext *codec_context) {int ret = 0;ret = avcodec_send_packet(codec_context, this->av_packet_);if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {return 0;}if (ret == AVERROR(EINVAL)) {throw std::runtime_error(vformat("avcodec_send_packet failure. error:%s", av_err2str(ret)));}if (ret == AVERROR_INPUT_CHANGED) {throw std::runtime_error(vformat("avcodec_send_packet failure. error:%s", av_err2str(ret)));}if (ret < 0) {throw std::runtime_error(vformat("avcodec_send_packet failure. error:%s", av_err2str(ret)));}while (ret >= 0) {ret = avcodec_receive_frame(codec_context, this->av_frame_);if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {return 0;}if (ret == AVERROR(EINVAL)) {throw std::runtime_error(vformat("avcodec_receive_failure. error:%s", av_err2str(ret)));}if (ret == AVERROR_INPUT_CHANGED) {throw std::runtime_error(vformat("avcodec_receive_failure failure. error:%s", av_err2str(ret)));}if (ret < 0) {return ret;}if (codec_context->codec_type == AVMEDIA_TYPE_AUDIO) {ret = OutputAudioFrame(this->av_frame_);} else if (codec_context->codec_type == AVMEDIA_TYPE_VIDEO) {ret = OutputVideoFrame(this->av_frame_);}av_frame_unref(this->av_frame_);// if dump frame failed , return ret;if (ret < 0) {return ret;}}return 0;
}int Demuxing::OutputAudioFrame(AVFrame *frame) {if (this->audio_swr_context_) {// https://ffmpeg.org/doxygen/2.4/resampling__audio_8c_source.htmluint8_t **audio_dst_data = nullptr;int linesize = 0;// 为输出分配缓存av_samples_alloc_array_and_samples(&audio_dst_data, &linesize, output_channels_,frame->nb_samples, AV_SAMPLE_FMT_S16, 1);// 转换int num_frames = swr_convert(audio_swr_context_,audio_dst_data,frame->nb_samples,const_cast<const uint8_t **>(frame->data),frame->nb_samples);int unpadded_line_size = num_frames * av_get_bytes_per_sample(AV_SAMPLE_FMT_S16) * output_channels_;std::cout << vformat("Write audio frame %d,size=%d", this->audio_frame_counter_++, unpadded_line_size)<< std::endl;output_audio_stream_.write(reinterpret_cast<const char *>(audio_dst_data[0]), unpadded_line_size);// 释放缓存if (&audio_dst_data[0]) {av_freep(&audio_dst_data[0]);}av_freep(&audio_dst_data);// ffplay -showmode 1 -f s16le -channels 2 -ar 44100 cmake-build-debug/audio.pcm} else {int unpadded_line_size = frame->nb_samples * av_get_bytes_per_sample(AVSampleFormat(frame->format));std::cout << vformat("Write audio frame %d,size=%d", this->audio_frame_counter_++, unpadded_line_size)<< std::endl;// flt32格式的话：frame->extended_data[0] == frame->data[0]for (int i = 0; i < unpadded_line_size; i += 4) {output_audio_stream_.write(reinterpret_cast<const char *>(frame->data[0] + i), 4);output_audio_stream_.write(reinterpret_cast<const char *>(frame->data[1] + i), 4);}}return 1;
}int Demuxing::OutputVideoFrame(AVFrame *frame) {if (frame->width != this->video_codec_context_->width|| frame->height != this->video_codec_context_->height|| frame->format != this->video_codec_context_->pix_fmt) {throw std::runtime_error("The video frame width,height and fmt must same .");}av_image_copy(src_image_data_,src_image_data_line_size_,const_cast<const uint8_t ** > ( reinterpret_cast< uint8_t **>(frame->data)),frame->linesize,this->video_codec_context_->pix_fmt,frame->width,frame->height);if (this->video_sws_context_) {int h = sws_scale(this->video_sws_context_,src_image_data_,src_image_data_line_size_,0,frame->height,dst_image_data_,dst_image_data_line_size_);output_video_stream_.write(reinterpret_cast<const char *>(this->dst_image_data_[0]), this->dst_image_data_buffer_size_);std::cout << vformat("sws_scale return %d",h) << std::endl;} else {std::cout << vformat("Write video frame %d,size=%d,width=%d,height=%d,fmt=%s",this->video_frame_counter_++,this->src_image_data_buffer_size_,frame->width,frame->height,av_get_pix_fmt_name(AVPixelFormat(frame->format)))<< std::endl;
// output_video_stream_.write(reinterpret_cast<const char *>(this->src_image_data_[0]), this->src_image_data_buffer_size_ );output_video_stream_.write(reinterpret_cast<const char *>(this->src_image_data_[0]), frame->width * frame->height );output_video_stream_.write(reinterpret_cast<const char *>(this->src_image_data_[1]), frame->width * frame->height / 4 );output_video_stream_.write(reinterpret_cast<const char *>(this->src_image_data_[2]), frame->width * frame->height / 4 );}return 1;
}/*** @param type 音频/视频*/
void Demuxing::OpenCodecContext(AVMediaType type, AVCodecContext **codec_context, int *stream_index) const {// find video*stream_index = av_find_best_stream(format_context_,type, -1,-1, nullptr, 0);if (*stream_index < 0) {// stream not foundthrow std::runtime_error(vformat("Find stream %s error :%s",av_get_media_type_string(type),av_err2str(*stream_index)));}AVStream *stream = format_context_->streams[*stream_index];if (stream == nullptr) {throw std::runtime_error("Find video stream failure.");}const AVCodec *codec = avcodec_find_decoder(stream->codecpar->codec_id);if (codec == nullptr) {throw std::runtime_error("Find video codec failure.");}*codec_context = avcodec_alloc_context3(codec);if (avcodec_parameters_to_context(*codec_context, stream->codecpar) < 0) {throw std::runtime_error("Fill parameters failure.");}if (avcodec_open2(*codec_context, codec, nullptr) < 0) {throw std::runtime_error("Open avcodec failure.");}
}void Demuxing::AllocSrcImage() {this->src_image_data_buffer_size_ = av_image_alloc(this->src_image_data_,this->src_image_data_line_size_,this->video_codec_context_->width,this->video_codec_context_->height,this->video_codec_context_->pix_fmt,1);if (this->src_image_data_buffer_size_ < 0) {throw std::runtime_error(vformat("Alloc image error,message:%s", av_err2str(this->src_image_data_buffer_size_)));}
}
void Demuxing::AllocDstImage() {this->dst_image_data_buffer_size_= av_image_alloc(this->dst_image_data_,this->dst_image_data_line_size_,this->video_codec_context_->width /2,this->video_codec_context_->height /2,this->video_codec_context_->pix_fmt,1);if (this->dst_image_data_buffer_size_ < 0) {throw std::runtime_error(vformat("Alloc image error,message:%s", av_err2str(this->src_image_data_buffer_size_)));}
}使用sws_scale转换视频、使用swr_convert转换音频相关推荐
- 【PC工具】更新：在线智能抠图工具，在线视频、图片、音频等转换工具，绿色免安装抠图神奇抠图工具...
今天分享两款简单粗暴的工具软件,在线工具,不用安装,只需收藏. 在线智能抠图工具: https://www.gaoding.com/koutu/ 先上传图片,然后和上次的软件一样,用鼠标划一划就能扣出 ...
- python批量音频转格式_GitHub - shede333/SWConvertVideoToAudio: Python批量转换 视频 为 音频MP3（即提取音频文件）...
Python批量转换 视频 为 音频MP3(即提取音频文件) 输入文件格式:ffmpeg支持的视频文件 输出格式格式:mp3文件 使用方法: 注意:使用前需要先安装 ffmpeg 才行(Python最 ...
- 怎样将腾讯视频qlv格式转换成mp3音频
很多人都用腾讯视频观看电影.电视及各种视频,很多时候,看到精彩的视频想把它们保存下来,并且能够像普通视频那样播放.腾讯视频有一个缓存视频的功能,但是视频下载之后,发现都是.qlv格式,只有腾讯视频播放 ...
- 使用 FFmpeg 转换视频/音频格式 | 开源 免费 | 不用套壳软件
在我的上篇文章中,只有 不到0.43% 的人点了赞 所以如果觉得做的还行的话可以给我点个赞.收个藏,这对我真的很重要!QWQ 注: FFmpeg 有很多功能,这里只用来转换格式. 前言 emm- 据说 ...
- 如何把视频的声音转换成音频？
如何把视频的声音转换成音频?小编是一名从事网络运营推广的上班族,随着视频营销的兴起,小编也开始制作视频然后发布到各大平台,想要做好视频营销,那么平时肯定需要收集各种素材,例如视频.音频.图片等素材,好 ...
- 【PC工具】在线格式转换工具，在线智能抠图工具，在线视频、图片、音频等转换工具，绿色免安装抠图神奇抠图工具
微信关注 "DLGG创客DIY" 设为"星标",重磅干货,第一时间送达. 今天分享两款简单粗暴的工具软件,在线工具,不用安装,只需收藏. gaoding在线智能 ...
- 【PC工具】更新整理在线格式转换工具：在线智能抠图工具，在线视频、图片、音频等转换工具，绿色免安装抠图神奇抠图工具...
今天分享两款简单粗暴的工具软件,在线工具,不用安装,只需收藏. gaoding在线智能抠图工具 cloudconvert在线格式转换 https://cloudconvert.com/pages-to ...
- 视频.m4s格式转换成mp4，m4s音频转mp3 blbl视频下载
在windows下操作,需要借助ffmpeg工具. 在这里下载工具,http://www.121down.com/soft/softview-103719.html#downaddress 下载后解压 ...
- MP4视频如何转成mp3？转换视频为音频的解决之道
什么是音频文件呢?我们印象中,全部那些可以听进耳朵里的其实都可以说是音频哦!包括其实我们平常在听的音乐文件格式也是,只能听,没办法看.主要是MP3文件,也有其他的类似AAC.WAV等等!有时候需要将一 ...
最新文章
- c# 字典按ascii 排序_利用工作表函数，对字典键进行排序并给出对应重复个数
- 13款基于jQuery Mobile的布局插件和示例
- java nio详解,Java NIO API详解
- 3.5 卷积神经网络进阶-Inception-mobile_net 实战
- 【a202】【9208】输油管道问题
- 光头强的圆球机器人视频_《熊出没狂野大陆》快上映了，看了多年光头强，还能有新鲜动画吗...
- 结构与表现分离的思想
- mysql 导出中间 数据_mysql导出数据库几种方法
- Python 快速搭建文件上传服务器
- 免费苹果账号（apple id）申请ios证书p12真机调试
- web服务器主机头文件,在Win2k下建立虚拟Web主机
- PMP题目与解题思路（第二天）
- SAP中采购收货冲销和退货适用情形简析
- Redhat8.0 安装web consol、可视化管理工具
- Kaggle下载criteo数据集
- JimuReport 1.4.0-beta 里程碑版本发布，免费的低代码报表
- python下载文件并改名_用 Python 给下载的 B 站视频文件批量改名
- numpy loadtxt错误ValueError: Wrong number of columns at line ***
- FireFox下Canvas使用图像合成绘制SVG的Bug
- 【Python】为图片加上数字上标