RIoTBoard开发板系列笔记（四）—

视频解码分为软解和硬解两种，简单的来说纯粹依靠CPU来解码的称为软解，而使用GPU/VPU解码的称为硬解。常规地直接用CPU解码会极大地消耗CPU的运算能力，硬件解码是通过显卡的视频加速功能对高清视频进行解码,能够将CPU从繁重的视频解码运算中释放出来。
RIoTBoard使用的芯片是i.mx6 solo，芯片中包含一个VPU （video processing unit），可以进行硬件编解码加速，心血来潮，折腾一下视频硬件解码加速。

NXP官方提供了一个libimxvpuapi lib封装了vpu的使用方法，libimxvpuapi 又依赖libimxdmabuffer，所以只要编译出libimxvpuapi和libimxdmabuffer，运行libimxvpuapi 的testcase 就可以实现硬件编解码。

1 交叉编译libimxdmabuffer

libimxdmabuffer代码下载地址：

git clone https://github.com/Freescale/libimxdmabuffer.git

git 仓库里的readme有简单的说明，支持那些平台、如何编译等等，可以进行参考。
编译前需要配置toolchain以及imx-linux-headers-path，imx-linux-headers-path指向BSP中的linux-imx/include/即可。

./waf configure CC=arm-none-linux-gnueabi-gcc \
--imx-linux-headers-path=/home/zhy/code/mypc/linux/linux-imx/include/ \
--prefix=/home/zhy/code/rIoTboard/libimxdmabuffer

配置的过程中出现了linux/mxcfb.h not found的error:

Setting top to                           : /home/zhy/code/mypc/gstreamer/third_lib/libimxdmabuffer
Setting out to                           : /home/zhy/code/mypc/gstreamer/third_lib/libimxdmabuffer/build
Checking for 'gcc' (C compiler)          : arm-none-linux-gnueabi-gcc
Checking for compiler switch -O2         : yes
Checking for compiler switch -DPIC       : yes
Checking for compiler switch -fPIC       : yes
Checking for compiler switch -pedantic   : yes
Checking for compiler switch -std=gnu99  : yes
Checking for compiler switch -Wall       : yes
Checking for compiler switch -Wextra     : yes
Checking for header linux/mxcfb.h        : not found
Could not find linux/mxcfb.h in path "/home/zhy/code/mypc/linux/linux-imx/include/" specified by --imx-linux-headers-path

查看log发现是这个头文件中uint未定义。

../../../../../../linux/linux-imx/include/linux/mxcfb.h:113:2: error: unknown type name 'uint'

解决方法把uint更改unsigned int 即可。

unsigned int flags;

重新配置，这次就可以配置成功，会输出如下log：

checking for linux/fb.h and the IPU header linux/ipu.h                    : yes
Checking for library g2d                                                  : not found
Checking for header g2d.h                                                 : not found
G2D not found (library found: 0 header found: 0); disabling G2D allocator
checking for linux/pxp_device.h                                           : no
linux/pxp_device.h was not found in i.MX linux headers path; disabling PxP allocator
libimxdmabuffer version 1.0.1
'configure' finished successfully (0.662s)

配置好后执行编译命令进行编译：

./waf

编译成功后显示如下log：

[5/8] Compiling libimxdmabuffer.pc.in
[6/8] Linking build/libimxdmabuffer.so
[7/8] Linking build/test-alloc
[8/8] Symlinking build/libimxdmabuffer.so
Waf: Leaving directory `/home/zhy/code/mypc/gstreamer/third_lib/libimxdmabuffer/build'
'build' finished successfully (0.422s)

执行install命令，install目录为prefix指定的路径：

./waf install

测试一下能不能链接到limxdmabuffer这个lib:

 arm-none-linux-gnueabi-gcc -L/home/zhy/code/rIoTboard/libimxdmabuffer/lib -limxdmabuffer

输出一下log就代表可以链接到，编译OK。

/usr/local/complie_tool/fsl-linaro-toolchain/bin/../arm-fsl-linux-gnueabi/multi-libs/default/usr/lib/crt1.o: In function `_start':
init.c:(.text+0x34): undefined reference to `main'
collect2: ld returned 1 exit status

最后将编译好的lib 打包一下，方便以后使用，这一步不是必须的。

tar czvf libimxdmabuffer.tar.gz libimxdmabuffer/

2 编译libimxvpuapi

编译好libimxdmabuffer就可以编译libimxvpuapi，首先下载代码下载：

git clone https://github.com/Freescale/libimxvpuapi.git

libimxvpuapi编译依赖libimxdmabuffer和libvpu这两个lib ,libimxdmabuffer已经编译好了，在libimxvpuapi目录新建一个third_lib文件夹，把libimxdmabuffer编译好的so和头文件放进取。至于libvpu的头文件和so,可以到开发板的里面去copy，需要copy ipu.h、vpu_io.h、vpu_lib.h这三个头文件以及libvpu.so、libvpu.so.4这两个so，同样把这写放到third_lib 下面。最后在third_lib/lib/pkgconfig目录下放上libimxdmabuffer.pc和 vpu_lib.pc这两个pkg-config文件，最终third_lib下面的目录结构：

├── include
│   ├── imxdmabuffer
│   │   ├── imxdmabuffer_config.h
│   │   ├── imxdmabuffer.h
│   │   ├── imxdmabuffer_ipu_allocator.h
│   │   └── imxdmabuffer_physaddr.h
│   ├── ipu.h
│   ├── vpu_io.h
│   └── vpu_lib.h
└── lib├── libimxdmabuffer.so -> libimxdmabuffer.so.1.0.1├── libimxdmabuffer.so.1 -> libimxdmabuffer.so.1.0.1├── libimxdmabuffer.so.1.0.1├── libvpu.a├── libvpu.so├── libvpu.so.4└── pkgconfig├── libimxdmabuffer.pc└── vpu_lib.pc

其中libimxdmabuffer.pc中内容，这里面的路径根据自己的目录可以进行修改：

prefix=/home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/third_lib
exec_prefix=/home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/third_lib
libdir=${prefix}/lib
includedir=${prefix}/includelibimxdmabuffer_ion=0Name: libimxdmabuffer
Description: library for allocating and managing physically contiguous memory ("DMA memory" or "DMA buffers") on i.MX devices
Version: 1.0.1
Libs: -L${libdir} -limxdmabuffer
Cflags: -I${includedir}

vpu_lib.pc中内容：

prefix=/home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/third_lib
exec_prefix=/home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/third_lib
libdir=${prefix}/lib
includedir=${prefix}/includeName: vpu
Description: libvpu
Version: 1.0.1
Libs: -L${libdir} -lvpu
Cflags: -I${includedir}

准备工作完成后进行编译配置：

./waf configure CC=arm-none-linux-gnueabi-gcc \
--sysroot-path=/home/zhy/code/mypc/linux/linux-imx/include/ \
--prefix=/home/zhy/code/rIoTboard/libimxvpuapi \
--imx-platform=imx6
--imx-headers=/home/zhy/code/mypc/linux/linux-imx/include/

配置的时候显示找不到libimxdmabuffer 这个lib:

Checking if this combination works       : yes
Checking for program 'pkg-config'        : /usr/bin/pkg-config
Checking for 'libimxdmabuffer >= 0.9.0'  : not found
The configuration failed
(complete log in /home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/build/config.log)

查看log 发现是没有找到libimxdmabuffer的pkg-config文件，

----------------------------------------
Checking for program 'pkg-config'
/usr/bin/pkg-config
find program=['pkg-config'] paths=['/usr/local/sbin', '/usr/local/bin', '/usr/sbin', '/usr/bin', '/sbin', '/bin', '/usr/games', '/usr/local/games', '/snap/bin', '/usr/local/complie_tool/fsl-linaro-toolchain/bin'] var='PKGCONFIG' -> ['/usr/bin/pkg-config']
----------------------------------------
Checking for 'libimxdmabuffer >= 0.9.0'
['/usr/bin/pkg-config', '--cflags', '--libs', 'libimxdmabuffer', '>=', '0.9.0']
err: Package libimxdmabuffer was not found in the pkg-config search path.
Perhaps you should add the directory containing `libimxdmabuffer.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libimxdmabuffer' found

通过PKG_CONFIG_PATH关键字增加pkg-config文件的路径即可。

./waf configure CC=arm-none-linux-gnueabi-gcc \
PKG_CONFIG_PATH=/home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/third_lib/lib/pkgconfig \
--imx-platform=imx6 \
--sysroot-path=/home/zhy/code/mypc/linux/linux-imx/include/ \
--prefix=/home/zhy/code/rIoTboard/libimxvpuapi \
--imx-headers=/home/zhy/code/mypc/linux/linux-imx/include/

再次配置，有以下log印出则代表配置成功：

Setting top to                           : /home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi
Setting out to                           : /home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/build
Checking for 'gcc' (C compiler)          : arm-none-linux-gnueabi-gcc
Checking for compiler switch -Wextra     : yes
Checking for compiler switch -Wall       : yes
Checking for compiler switch -pedantic   : yes
Checking for compiler switch -fPIC       : yes
Checking for compiler switch -DPIC       : yes
Checking for compiler switch -O2         : yes
Checking for compiler switch -std=gnu99  : yes
Now checking the combination of CFLAGS -Wextra -Wall -pedantic -fPIC -DPIC -O2 -std=gnu99
Checking if this combination works       : yes
Checking for compiler switch -std=c99    : yes
Now checking the combination of CFLAGS -Wextra -Wall -pedantic -fPIC -DPIC -O2 -std=c99
Checking if this combination works       : yes
Checking for program 'pkg-config'        : /usr/bin/pkg-config
Checking for 'libimxdmabuffer >= 0.9.0'  : yes
Using "/home/zhy/code/mypc/linux/linux-imx/include" as sysroot path
Checking for library vpu                 : yes
checking if ENC_ENABLE_SOF_STUFF exists  : no
i.MX linux headers path: /home/zhy/code/mypc/linux/linux-imx/include/
checking for the IPU header linux/ipu.h  : yes
libimxvpuapi version 2.2.0
'configure' finished successfully (0.692s)

执行以下命令进行编译

./waf

编译时出现了’IPU_PIX_FMT_NV16’ undeclared的error。

../imxvpuapi2/imxvpuapi2_imx6_coda_ipu.c: In function 'get_ipu_pixel_format':
../imxvpuapi2/imxvpuapi2_imx6_coda_ipu.c:33:76: error: 'IPU_PIX_FMT_NV16' undeclared (first use in this function)

ipu.h里面没有定义这个format，目前不需要NV12这个color format，在get_ipu_pixel_format函数中注释掉IPU_PIX_FMT_NV16 这个color_format即可。

static uint32_t get_ipu_pixel_format(ImxVpuApiColorFormat color_format)
{switch (color_format){case IMX_VPU_API_COLOR_FORMAT_FULLY_PLANAR_YUV420_8BIT: return IPU_PIX_FMT_YUV420P;case IMX_VPU_API_COLOR_FORMAT_SEMI_PLANAR_YUV420_8BIT:  return IPU_PIX_FMT_NV12;case IMX_VPU_API_COLOR_FORMAT_FULLY_PLANAR_YUV411_8BIT: return IPU_PIX_FMT_YUV410P;case IMX_VPU_API_COLOR_FORMAT_FULLY_PLANAR_YUV422_HORIZONTAL_8BIT: return IPU_PIX_FMT_YUV422P;//case IMX_VPU_API_COLOR_FORMAT_SEMI_PLANAR_YUV422_HORIZONTAL_8BIT: return IPU_PIX_FMT_NV16;case IMX_VPU_API_COLOR_FORMAT_FULLY_PLANAR_YUV444_8BIT: return IPU_PIX_FMT_YUV444P;case IMX_VPU_API_COLOR_FORMAT_YUV400_8BIT: return IPU_PIX_FMT_GREY;default: return 0;}
}

修改后编译成功，会有以下log印出，并在build 目录会生成libimxvpuapi2.so。

[14/19] Linking build/libimxvpuapi2.so
[15/19] Linking build/example/decode-example
[16/19] Linking build/example/encode-example
[17/19] Linking build/example/jpeg-dec-example
[18/19] Linking build/example/jpeg-enc-example
[19/19] Symlinking build/libimxvpuapi2.so
Waf: Leaving directory `/home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/build'
'build' finished successfully (1.970s)

测试一下能否动态链接到libimxvpuapi：

 arm-none-linux-gnueabi-gcc -L/home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/build -limxvpuapi2

有以下log印出则代表链接OK:

zhy@zhy-ThinkPad-E480:~/code/mypc/gstreamer/third_lib/libimxvpuapi/build$  arm-none-linux-gnueabi-gcc -L/home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/build -limxvpuapi2
/usr/local/complie_tool/fsl-linaro-toolchain/bin/../lib/gcc/arm-fsl-linux-gnueabi/4.6.2/../../../../arm-fsl-linux-gnueabi/bin/ld: warning: libimxdmabuffer.so.1, needed by /home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/build/libimxvpuapi2.so, not found (try using -rpath or -rpath-link)
/usr/local/complie_tool/fsl-linaro-toolchain/bin/../arm-fsl-linux-gnueabi/multi-libs/default/usr/lib/crt1.o: In function `_start':
init.c:(.text+0x34): undefined reference to `main'

将编译好的lib进行打包，以便后续使用，这一部分为非必要步骤。

mkdir libimxvpuapi
mkdir -p libimxvpuapi/lib libimxvpuapi/include/imxvpuapi2 libimxvpuapi/pkgconfig
cp -rf  build/*.so.* libimxvpuapi/lib/
cp -rf  imxvpuapi2/*.h libimxvpuapi/include/imxvpuapi2
cp -rf build/libimxvpuapi2.pc libimxvpuapi/lib/pkgconfig

然后在更改一下libimxvpuapi2.pc ，将libdir和includedir改成可根据prefix动态配置。

prefix=/home/zhy/code/rIoTboard/libimxvpuapi
exec_prefix=/home/zhy/code/rIoTboard/libimxvpuapi
libdir=${prefix}/lib
includedir=${prefix}/include

最后打包lib:

tar czvf libimxvpuapi.tar.gz libimxvpuapi/

3 开发板上测试使用vpu解码

libimxvpuapi编译好后在build/example 下面后生成可执行文件，将example整个文件夹复制到开放板上，同时还要复制libimxvpuapi和libimxdmabuffer编译好的so到开放板的/lib/目录下，下载一个h264码流，用mediainfo看一下信息：

把这个码流也复制到开发板上libimxvpuapi/example目录，在example执行以测试使用vpu解码h264。

./decode-example -i ./ds.h264 -o out.yuv

用手机拍了一下执行结果：

把解码后的out.yuv复制到PC 上，再用mediainfo查看，已经解码成yuv数据了，解码前190kb，解码后18.7M。

用VLC也可以播放out.yuv：