1.介绍：
MediaCodec类可用于访问Android底层的媒体编解码器，也就是，编码器/解码器组件。它是Android底层多媒体支持基本架构的一部分（通常与MediaExtractor, MediaSync, MediaMuxer, MediaCrypto, MediaDrm, Image, Surface, 以及AudioTrack一起使用）；MediaCodec作为比较年轻的Android多媒体硬件编解码框架，在终端硬解方案中带来了很大便利。Android源码中的CTS部分也给出了很多可以关于Media编解码的Demo。

2.解码
Android的MediaCodec解码需要分为视频、音频解码。

首先获取MediaCodec支持的数量，根据MediaCodec的句柄获取MediaCodec支持的编解码格式

MediaCodecList.getCodecCount()

MediaCodecList.getCodecInfoAt(i);

比如通过以下测试代码，就可以知道终端MediaCodec的解码能力：

        int n = MediaCodecList.getCodecCount();for (int i = 0; i < n; ++i) {MediaCodecInfo info = MediaCodecList.getCodecInfoAt(i);String[] supportedTypes = info.getSupportedTypes();boolean mime_support = false;if(info.isEncoder()){return;}for (int j = 0; j < supportedTypes.length; ++j) {Log.d(TAG, "codec info:" + info.getName()+" supportedTypes:" + supportedTypes[j]);if (supportedTypes[j].equalsIgnoreCase(mime)) {mime_support = true;}}}

code
surpport

OMX.amlogic.hevc.decoder.awesome
video/hevc

OMX.amlogic.avc.decoder.awesome
video/avc

OMX.amlogic.mpeg4.decoder.awesome
video/mp4v-es

OMX.amlogic.h263.decoder.awesome
video/3gpp

OMX.amlogic.mpeg2.decoder.awesome
video/mpeg2

OMX.amlogic.vc1.decoder.awesome
video/vc1

OMX.amlogic.vc1.decoder.awesome
video/wvc1

OMX.amlogic.wmv3.decoder.awesome
video/wmv3

OMX.amlogic.mjpeg.decoder.awesome
video/mjpeg

OMX.google.amrnb.decoder
audio/3gpp

OMX.google.amrwb.decoder
audio/amr-wb

OMX.google.aac.decoder
audio/mp4a-latm

OMX.google.adif.decoder
audio/aac-adif

OMX.google.latm.decoder
audio/aac-latm

OMX.google.adts.decoder
audio/adts

OMX.google.g711.alaw.decoder
audio/g711-alaw

OMX.google.g711.mlaw.decoder
audio/g711-mlaw

OMX.google.adpcm.ima.decoder
audio/adpcm-ima

OMX.google.adpcm.ms.decoder
audio/adpcm-ms

OMX.google.vorbis.decoder
audio/vorbis

OMX.google.alac.decoder
audio/alac

OMX.google.wma.decoder
audio/wma

OMX.google.wmapro.decoder
audio/wmapro

OMX.google.ape.decoder
audio/ape

OMX.google.truehd.decoder
audio/truehd

OMX.google.ffmpeg.decoder
audio/ffmpeg

OMX.google.raw.decoder
audio/raw

OMX.google.mpeg4.decoder
video/mp4v-es

OMX.google.h263.decoder
video/3gpp

OMX.google.h264.decoder
video/avc

OMX.google.vp8.decoder
video/x-vnd.on2.vp8

OMX.google.vp9.decoder
video/x-vnd.on2.vp9

OMX.google.vp6.decoder
video/x-vnd.on2.vp6

OMX.google.vp6a.decoder
video/x-vnd.on2.vp6a

OMX.google.vp6f.decoder
video/x-vnd.on2.vp6f

OMX.google.rm10.decoder
video/rm10

OMX.google.rm20.decoder
video/rm20

OMX.google.rm40.decoder
video/rm40

OMX.google.wmv2.decoder
video/wmv2

OMX.google.wmv1.decoder
video/wmv1

AML.google.ac3.decoder
audio/ac3

AML.google.ec3.decoder
audio/eac3

OMX.google.mp2.decoder
audio/mpeg-L2

OMX.google.mp3.decoder
audio/mpeg

AML.google.dtshd.decoder
audio/dtshd

OMX.google.raw.decoder
audio/raw

OMX.google.vp6.decoder
video/x-vnd.on2.vp6

OMX.google.vp6a.decoder
video/x-vnd.on2.vp6a

OMX.google.vp6f.decoder
video/x-vnd.on2.vp6f

OMX.google.h265.decoder
video/hevc

OMX.google.wmv2.decoder
video/wmv2

OMX.google.wmv2.decoder
video/wmv1
Android提供了MediaExtractor来分离本地/网络视频流的音视频。

首先定义了一个统一音视频处理的类：

public class MediaDecoder extends Thread{protected String mVideoFilePath = null; public static final long TIME_US = 10000;protected MediaExtractor mExtractor = null;protected MediaCodec mDecoder = null;protected MediaFormat mediaFormat;protected UpstreamCallback mCallback;protected Surface mSurface;public MediaDecoder(String videoFilePath, Surface surface,UpstreamCallback callback) {this.mVideoFilePath = videoFilePath;this.mSurface = surface;this.mCallback = callback;}@Overridepublic void run() {// TODO Auto-generated method stubsuper.run();prepare();}public void prepare(){try {File videoFile = new File(mVideoFilePath);mExtractor = new MediaExtractor();mExtractor.setDataSource(videoFile.toString());} catch (IOException e) {// TODO Auto-generated catch blocke.printStackTrace();}}}

① 视频解码：
首先创建视频的解码器：

            for (int i = 0; i < mExtractor.getTrackCount(); i++) {MediaFormat format = mExtractor.getTrackFormat(i);String mime = format.getString(MediaFormat.KEY_MIME);if (mime.startsWith("video/")) {mExtractor.selectTrack(i);mDecoder = MediaCodec.createDecoderByType(mime);if(mCallback != null){mDecoder.configure(format, null, null, 0);    //decode flag no output for surface}else{mDecoder.configure(format, mSurface, null, 0);    //decode flag output to surface}break;}}if (mDecoder == null) {Log.e(TAG, "Can't find video info!");return;}mDecoder.start();

当MediaCodec的解码buffer空闲时（mDecoder.dequeueInputBuffer），就可以把分离出的视频数据填充到buffer中（mDecoder.queueInputBuffer），让Decoder开始解码：

                    int inIndex = mDecoder.dequeueInputBuffer(TIME_US);if (inIndex >= 0) {ByteBuffer buffer = inputBuffers[inIndex];int sampleSize = mExtractor.readSampleData(buffer, 0);if (sampleSize < 0) {// We shouldn't stop the playback at this point, just pass the EOS// flag to mDecoder, we will get it again from the// dequeueOutputBufferLog.d(TAG, "InputBuffer BUFFER_FLAG_END_OF_STREAM");mDecoder.queueInputBuffer(inIndex, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM);isEOS = true;} else {mDecoder.queueInputBuffer(inIndex, 0, sampleSize, mExtractor.getSampleTime(), 0);mExtractor.advance();}}

Decoder解码出来的数据（mDecoder.dequeueOutputBuffer），解码的数据就可以做该做的处理，比如再编码，合成文件等。

            BufferInfo info = new BufferInfo();int outIndex = mDecoder.dequeueOutputBuffer(info, TIME_US);switch (outIndex) {case MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED:Log.d(TAG, "INFO_OUTPUT_BUFFERS_CHANGED");outputBuffers = mDecoder.getOutputBuffers();break;case MediaCodec.INFO_OUTPUT_FORMAT_CHANGED:Log.d(TAG, "New format " + mDecoder.getOutputFormat());break;case MediaCodec.INFO_TRY_AGAIN_LATER:Log.d(TAG, "dequeueOutputBuffer timed out!");break;default://here erro?Log.d(TAG, "outIndex:"+outIndex);ByteBuffer buffer = outputBuffers[outIndex];Log.d(TAG, "ByteBuffer limit:"+buffer.limit()+" info size:"+info.size);final byte[] chunk = new byte[info.size];buffer.get(chunk);if(mCallback != null){//mCallback.UpstreamCallback(chunk,info.size);}//clear buffer,otherwise get the same buffer which is the last bufferbuffer.clear();if(DEBUG_VIDEO)Log.v(TAG, "We can't use this buffer but render it due to the API limit, " + buffer);// We use a very simple clock to keep the video FPS, or the video// playback will be too fastwhile (info.presentationTimeUs / 1000 > System.currentTimeMillis() - startMs) {try {sleep(10);} catch (InterruptedException e) {e.printStackTrace();break;}}mDecoder.releaseOutputBuffer(outIndex, true);break;}

视频解码的完整代码：

public class VideoDecoder extends MediaDecoder{
private static final String TAG = “VideoDecode”;
private static final boolean DEBUG_VIDEO = false;

public VideoDecoder(String videoFilePath,Surface surface,UpstreamCallback callback){super(videoFilePath, surface, callback);
}@Override
public void run() {// TODO Auto-generated method stubsuper.run();VideoDecodePrepare();
}public void VideoDecodePrepare() {  try {              for (int i = 0; i < mExtractor.getTrackCount(); i++) {MediaFormat format = mExtractor.getTrackFormat(i);String mime = format.getString(MediaFormat.KEY_MIME);if (mime.startsWith("video/")) {mExtractor.selectTrack(i);mDecoder = MediaCodec.createDecoderByType(mime);if(mCallback != null){mDecoder.configure(format, null, null, 0);  //decode flag no output for surface}else{mDecoder.configure(format, mSurface, null, 0);  //decode flag output to surface}break;}}if (mDecoder == null) {Log.e(TAG, "Can't find video info!");return;}mDecoder.start();  ByteBuffer[] inputBuffers = mDecoder.getInputBuffers();ByteBuffer[] outputBuffers = mDecoder.getOutputBuffers();boolean isEOS = false;long startMs = System.currentTimeMillis();while (!Thread.interrupted()) {if (!isEOS) {int inIndex = mDecoder.dequeueInputBuffer(TIME_US);if (inIndex >= 0) {ByteBuffer buffer = inputBuffers[inIndex];int sampleSize = mExtractor.readSampleData(buffer, 0);if (sampleSize < 0) {// We shouldn't stop the playback at this point, just pass the EOS// flag to mDecoder, we will get it again from the// dequeueOutputBufferLog.d(TAG, "InputBuffer BUFFER_FLAG_END_OF_STREAM");mDecoder.queueInputBuffer(inIndex, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM);isEOS = true;} else {mDecoder.queueInputBuffer(inIndex, 0, sampleSize, mExtractor.getSampleTime(), 0);mExtractor.advance();}}}BufferInfo info = new BufferInfo();int outIndex = mDecoder.dequeueOutputBuffer(info, TIME_US);switch (outIndex) {case MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED:Log.d(TAG, "INFO_OUTPUT_BUFFERS_CHANGED");outputBuffers = mDecoder.getOutputBuffers();break;case MediaCodec.INFO_OUTPUT_FORMAT_CHANGED:Log.d(TAG, "New format " + mDecoder.getOutputFormat());break;case MediaCodec.INFO_TRY_AGAIN_LATER:Log.d(TAG, "dequeueOutputBuffer timed out!");break;default://here erro?Log.d(TAG, "outIndex:"+outIndex);ByteBuffer buffer = outputBuffers[outIndex];Log.d(TAG, "ByteBuffer limit:"+buffer.limit()+" info size:"+info.size);final byte[] chunk = new byte[info.size];buffer.get(chunk);if(mCallback != null){//mCallback.UpstreamCallback(chunk,info.size);}//clear buffer,otherwise get the same buffer which is the last bufferbuffer.clear();if(DEBUG_VIDEO)Log.v(TAG, "We can't use this buffer but render it due to the API limit, " + buffer);// We use a very simple clock to keep the video FPS, or the video// playback will be too fastwhile (info.presentationTimeUs / 1000 > System.currentTimeMillis() - startMs) {try {sleep(10);} catch (InterruptedException e) {e.printStackTrace();break;}}mDecoder.releaseOutputBuffer(outIndex, true);break;}// All decoded frames have been rendered, we can stop playing nowif ((info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) {Log.d(TAG, "OutputBuffer BUFFER_FLAG_END_OF_STREAM");break;}}mDecoder.stop();mDecoder.release();mExtractor.release();} catch (Exception ioe) {  Log.d(TAG,"failed init decoder", ioe);  }
}

注意video解码时，如果使用surface来输出，则outbuffer会被消耗掉，想要在视频输出的同时转码，建议使用OpenESGL来绘制窗口（这部分后面提供代码），保留原有buffer。
② 音频解码：
基本和视频解码的方式一样，只不过是通过MediaExtractor从文件中分离出音频，然后使用指定的音频MediaFormat来通知Decoder解码。

音频解码器的创建如下：

          for (int i = 0; i < mExtractor.getTrackCount(); i++) {MediaFormat format = mExtractor.getTrackFormat(i);String mime = format.getString(MediaFormat.KEY_MIME);if (mime.startsWith("audio/")) {mExtractor.selectTrack(i);mSampleRate = format.getInteger(MediaFormat.KEY_SAMPLE_RATE);channel = format.getInteger(MediaFormat.KEY_CHANNEL_COUNT);mDecoder = MediaCodec.createDecoderByType(mime);mDecoder.configure(format, null, null, 0);break;}}if (mDecoder == null) {Log.e(TAG, "Can't find audio info!");return;}mDecoder.start();

在这里，我直接通过Android的AudioTrack来输出音频，完整代码如下：

public class AudioDecoder extends MediaDecoder {
private static final String TAG = “VideoDecode”;
private static final boolean DEBUG_AUDIO = false;

private int mSampleRate = 0;
private int channel = 0;public AudioDecoder(String videoFilePath, Surface surface,UpstreamCallback callback) {super(videoFilePath, surface, callback);
}@Override
public void run() {// TODO Auto-generated method stubsuper.run();AudioDecodePrepare();
}public void AudioDecodePrepare() {try {for (int i = 0; i < mExtractor.getTrackCount(); i++) {MediaFormat format = mExtractor.getTrackFormat(i);String mime = format.getString(MediaFormat.KEY_MIME);if (mime.startsWith("audio/")) {mExtractor.selectTrack(i);mSampleRate = format.getInteger(MediaFormat.KEY_SAMPLE_RATE);channel = format.getInteger(MediaFormat.KEY_CHANNEL_COUNT);mDecoder = MediaCodec.createDecoderByType(mime);mDecoder.configure(format, null, null, 0);break;}}if (mDecoder == null) {Log.e(TAG, "Can't find audio info!");return;}mDecoder.start();ByteBuffer[] inputBuffers = mDecoder.getInputBuffers();ByteBuffer[] outputBuffers = mDecoder.getOutputBuffers();BufferInfo info = new BufferInfo();int buffsize = AudioTrack.getMinBufferSize(mSampleRate,AudioFormat.CHANNEL_OUT_STEREO,AudioFormat.ENCODING_PCM_16BIT);AudioTrack audioTrack = new AudioTrack(AudioManager.STREAM_MUSIC,mSampleRate, AudioFormat.CHANNEL_OUT_STEREO,AudioFormat.ENCODING_PCM_16BIT, buffsize,AudioTrack.MODE_STREAM);audioTrack.play();boolean isEOS = false;long startMs = System.currentTimeMillis();while (!Thread.interrupted()) {if (!isEOS) {int inIndex = mDecoder.dequeueInputBuffer(TIME_US);if (inIndex >= 0) {ByteBuffer buffer = inputBuffers[inIndex];int sampleSize = mExtractor.readSampleData(buffer, 0);if (sampleSize < 0) {// We shouldn't stop the playback at this point,// just pass the EOS// flag to mediaDecoder, we will get it again from// the// dequeueOutputBufferLog.d(TAG, "InputBuffer BUFFER_FLAG_END_OF_STREAM");mDecoder.queueInputBuffer(inIndex, 0, 0, 0,MediaCodec.BUFFER_FLAG_END_OF_STREAM);isEOS = true;} else {mDecoder.queueInputBuffer(inIndex, 0,sampleSize, mExtractor.getSampleTime(), 0);mExtractor.advance();}}}int outIndex = mDecoder.dequeueOutputBuffer(info, TIME_US);switch (outIndex) {case MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED:Log.d(TAG, "INFO_OUTPUT_BUFFERS_CHANGED");outputBuffers = mDecoder.getOutputBuffers();break;case MediaCodec.INFO_OUTPUT_FORMAT_CHANGED:MediaFormat format = mDecoder.getOutputFormat();Log.d(TAG, "New format " + format);audioTrack.setPlaybackRate(format.getInteger(MediaFormat.KEY_SAMPLE_RATE));break;case MediaCodec.INFO_TRY_AGAIN_LATER:Log.d(TAG, "dequeueOutputBuffer timed out!");break;default:ByteBuffer buffer = outputBuffers[outIndex];if(DEBUG_AUDIO)Log.v(TAG,"We can't use this buffer but render it due to the API limit, "+ buffer);final byte[] chunk = new byte[info.size];buffer.get(chunk);if(mCallback != null){mCallback.UpstreamCallback(chunk,info.size);}//clear buffer,otherwise get the same buffer which is the last bufferbuffer.clear();                 // We use a very simple clock to keep the video FPS, or the// audio playback will be too fastwhile (info.presentationTimeUs / 1000 > System.currentTimeMillis() - startMs) {try {sleep(10);} catch (InterruptedException e) {e.printStackTrace();break;}}// AudioTrack write dataaudioTrack.write(chunk, info.offset, info.offset+ info.size);mDecoder.releaseOutputBuffer(outIndex, false);break;}// All decoded frames have been rendered, we can stop playing nowif ((info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) {Log.d(TAG, "OutputBuffer BUFFER_FLAG_END_OF_STREAM");break;}}mDecoder.stop();mDecoder.release();mExtractor.release();audioTrack.stop();audioTrack.release();} catch (Exception ioe) {throw new RuntimeException("failed init encoder", ioe);}
}

}

通过以上处理，就可以使用硬解方案实现音视频播放了。
3.结束语
解码的知识先介绍这么多，下一篇会介绍MediaCodec来实现编码。感谢持续关注！

MediaCodec之Decoder相关推荐

  1. Android媒体解码MediaCodec,MediaExtractor

    Android提供了MediaPlayer播放器播放媒体文件,其实MediaPlyer只是对Android Media包下的MediaCodec和MediaExtractor进行了包装,方便使用.但是 ...

  2. Android OpenGL ES 学习(十二) - MediaCodec + OpenGL 解析H264视频+滤镜

    OpenGL 学习教程 Android OpenGL ES 学习(一) – 基本概念 Android OpenGL ES 学习(二) – 图形渲染管线和GLSL Android OpenGL ES 学 ...

  3. 安卓开发首次创建项目一直转圈_Android视频开发进阶(part3Android的Media API)

    秦子帅明确目标,每天进步一点点..... 作者 |  qing的世界地址 |  juejin.im/post/6844903574837657614 上两期我们已经学习了关于视频播放的基础知识,还有容 ...

  4. Android视频滤镜添加硬解码方案

    由于工作的需求,研究过了一段时间的Android 的音视频播放渲染以及编辑方面的知识,这里就自己一些浅薄的了解对所了解做一个简单的介绍和记录,如有不对的地方请指正!同时也会记录下硬件解码的情况下完成滤 ...

  5. android amr-wb 编解码

    平台  PX30 + Android 9.0 + AndroidStudio 4.1.3 概述  在Android 平台上实现AMR-WB的编解码, 要求不高, JAVA也行, C/CPP也行, 可惜 ...

  6. Android Graphics - Architecture

    说在前面的话 这篇文章翻译自Google 的Graphic Architecture. https://source.android.com/devices/graphics/architecture ...

  7. MediaCodeC解码视频指定帧，迅捷、精确

    原创文章,转载请联系作者 若待明朝风雨过,人在天涯!春在天涯 原文地址 提要 最近在整理硬编码MediaCodec相关的学习笔记,以及代码文档,分享出来以供参考.本人水平有限,项目难免有思虑不当之处, ...

  8. Android平台MediaCodec避坑指北

    https://www.jianshu.com/p/5d62a3cf0741 最近使用MediaCodec做编解码H264,写一点东西以免自己再次掉坑. 先说一下具体环境,使用的是,Windows10 ...

  9. MediaCodec在Android视频硬解码组件的应用

    https://yq.aliyun.com/articles/632892 云栖社区> 博客列表> 正文 MediaCodec在Android视频硬解码组件的应用 cheenc 2018- ...

  10. Android PC投屏简单尝试(录屏直播)2—硬解章(MediaCodec+RMTP)

    代码地址 :https://github.com/deepsadness/MediaProjectionDemo 想法来源 上一边文章的最后说使用录制的Api进行录屏直播.本来这边文章是预计在5月份完 ...

最新文章

  1. python编辑器，作为初学者该如何抉择？
  2. 比PCA更好用的监督排序—LDA分析、作图及添加置信-ggord
  3. Oracle表空间文件损坏后的排查及解决
  4. 初学者自学python要看什么书-初学者如何学习Python？掌握这17个实用小技巧快速入门！...
  5. 【机器视觉】 else算子
  6. 【51nod - 1875】 丢手绢（约瑟夫问题，可打表，用STL模拟）
  7. Python爬虫番外篇之Cookie和Session
  8. tomcat 转发 http接口的绝对路径文件
  9. linux 查看运行 job,如何通过Web查看job的运行情况
  10. linux udhcpc指令,linux下udhcpc的使用
  11. 记录：图片转字符画及文字转字符画
  12. 在网吧想免费上网又何不自己动动手呢？
  13. win10 无法识别x64dbg 插件
  14. Days14 ContentProvider ContentResolver
  15. 重温设计模式二 设计原则之依赖倒置原则
  16. Zxing.jar下载
  17. 智能多模式，视线追踪控制界面的应用（翻译）
  18. 阿里巴巴1688诚信通通过市场全面分析选品策略
  19. 苹果桌面主题_看腻了手机自带的桌面主题，试试这个
  20. linux中,＜rc.d＞,＜rc.local＞等带有rc的文件或目录 含义

热门文章

  1. 由于这台计算机上储存的远程桌面,“由于这台计算机没有远程桌面客户端访问许可证，远程会话被中断”的解决方案...
  2. 实验吧 天网管理系统
  3. vue前端开发微信支付和支付宝支付
  4. 【redis集群：2. 集群伸缩】
  5. Java自学经验分享
  6. Photoshop技术学习有感
  7. 《大型网站技术架构》《K8S进阶实战》等书籍！送45本！
  8. 提升用户体验？指示性设计元素不可或缺
  9. 禅道----产品经理创建项目集和产品线
  10. 基于微信跳蚤市场二手交易小程序系统设计与实现 开题报告