图像处理-RGB24转YUV420遇到的坑以及执行效率对比

感兴趣可以加QQ群85486140，大家一起交流相互学习下！

文章目录

一、色彩空间简介及一些参考文档
二、YUV和RGB转换公式
三、RGB和YUV420转换提前需要了解的知识
- 1.YUV420内存布局
- 2.RGB内存布局
- 3.转换方式
四、源代码执行效率对比
五、编译过程中遇到的问题

一、色彩空间简介及一些参考文档

不同的色彩空间，颜色的表现形式不同(见http://colorizer.org/　这个网站中简单介绍了各个色彩空间的模型。)。色彩空间中的颜色是可以相互转化的。我们常用的颜色空间有YUV和RGB色彩空间。

色彩空间简介：http://colorizer.org/
参考文章:1：一种简单的YUV转RGB的优化算法（之前微信推送的一篇文章）
参考文章2：YUV <——> RGB 转换算法（此文章分析的非常全面，并有github源码仓库，优先推荐参考）
JPG转RGB格式的网站：https://convertio.co/zh/jpg-rgb/

二、YUV和RGB转换公式

如果是RGB24的话，则它们和YUV的对应公式如下所示：

1.RGB转YUV

Y = 0.299R + 0.587G + 0.114B
U = -0.147R - 0.289G + 0.436B
V = 0.615R - 0.515G - 0.100B

2.YUV转RGB

R = Y + 1.14V
G = Y - 0.39U - 0.58V
B = Y + 2.03U

上面可以可以看到有浮点运算，这会导致耗时很长。下面的优化也是基于浮点运算，来的。

三、RGB和YUV420转换提前需要了解的知识

1.YUV420内存布局

                   W+--------------------+|Y0Y1Y2Y3...         ||...                 ||                    |   H|                    ||                    ||                    |+--------------------+|U0U1...   ||...       |   H/2|          |+----------+|V0V1...   ||...       |  H/2|          |+----------+w/2

2.RGB内存布局

虽然名字上叫RGB,但是排列的时候是按着BGR的顺序排列的，如下所示。

                        B                               G                               R+-----------------------------------------------------------------------------------------------+高字节 | B | B | B | B | B | B | B | B | G | G | G | G | G | G | G | G | R | R | R | R | R | R | R | R |   低字节+-----------------------------------------------------------------------------------------------+0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15  16  17  18  19  20  21  22  23

3.转换方式

转换的时候由于不是RGB24转YUV444(一对一)，而是转成YUV420.所以必然有些数据时要舍弃的。下面即将处理的图片分辨率是320 x 240的大小，所以RGB24的格式就会产生320 * 240 *3 byte的数据即230400，下图也能看到后缀为rgb图片的大小。对应YUV420图片大小则为320 * 240 * 1.5=115200 Byte的大小。同样下面实际生成的图片也是这么大。

RGB的存放方式如下所示，例如8*6像素的RGB图像，其数据分布式这样存放的。

           pixe0       pixe1      pixe2       pixe3       pixe4      pixe5        pixe6       pixe7+-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------+
Line0  | B | G | R | B | G | R | B | G | R | B | G | R | B | G | R | B | G | R | B | G | R | B | G | R | +-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------+
Line1  | B | G | R | B | G | R | B | G | R | B | G | R | B | G | R | B | G | R | B | G | R | B | G | R | +-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------+
Line2  | B | G | R | B | G | R | B | G | R | B | G | R | B | G | R | B | G | R | B | G | R | B | G | R | +-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------+
Line3  | B | G | R | B | G | R | B | G | R | B | G | R | B | G | R | B | G | R | B | G | R | B | G | R | +-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------+
Line4  | B | G | R | B | G | R | B | G | R | B | G | R | B | G | R | B | G | R | B | G | R | B | G | R | +-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------+
Line5  | B | G | R | B | G | R | B | G | R | B | G | R | B | G | R | B | G | R | B | G | R | B | G | R | +-----------|-----------|-----------|-----------|-----------|-----------|-----------|-----------+

那么由于UV是每4个点对应一组UV,则选择哪个点转换为U，那个点转换为V。针对这样的block选择每个block的左上角(pix0)像素转换成U，左下角(pix2)的转换成V，其实这4个点的像素都可以用来转换UV，只是大家都习惯这样来转。

|pix0|pix1|
|pix2|pix3|

其它网友YUV420和YUV444相互转换的的好博客：https://blog.csdn.net/guyuealian/article/details/82454945

四、源代码执行效率对比

VS2015源码包可以在这里下载：https://download.csdn.net/download/armwind/11589508
由于原始的转换过程中包含了浮点和乘法运算，所以需要优化算法。下面例程中也包含了优化前后的对比。

1.源码

#include "stdafx.h"
#include "iostream"
#include<Windows.h>
#include <stdio.h>using namespace::std;static void RGBToYUV_v1(int R, int G, int B, int* Y, int* U, int* V) {*Y = 0.299*R + 0.587*G + 0.114*B;*U = -0.147*R - 0.289*G + 0.436*B;*V = 0.615*R - 0.515*G - 0.100*B;
}static void YUVToRGB_v1(int Y, int U, int V, int* R, int* G, int* B) {*R = Y + 1.14*V;*G = Y - 0.39*U - 0.58*V;*B = Y + 2.03*U;
}static void RGBToYUV(int Red, int Green, int Blue, int* Y, int* U, int* V){*Y = ((Red << 6) + (Red << 3) + (Red << 2) + Red + (Green << 7) + (Green << 4) + (Green << 2) + (Green << 1) + (Blue << 4) + (Blue << 3) + (Blue << 2) + Blue) >> 8;*U = (-((Red << 5) + (Red << 2) + (Red << 1)) - ((Green << 6) + (Green << 3) + (Green << 1)) + ((Blue << 6) + (Blue << 5) + (Blue << 4))) >> 8;*V = ((Red << 7) + (Red << 4) + (Red << 3) + (Red << 2) + (Red << 1) - ((Green << 7) + (Green << 2)) - ((Blue << 4) + (Blue << 3) + (Blue << 1))) >> 8;
}static void YUVToRGB(int Y, int U, int V, int* Red, int* Green, int* Blue)
{*Red = ((Y << 8) + ((V << 8) + (V << 5) + (V << 2))) >> 8;*Green = ((Y << 8) - ((U << 6) + (U << 5) + (U << 2)) - ((V << 7) + (V << 4) + (V << 2) + V)) >> 8;*Blue = ((Y << 8) + (U << 9) + (U << 3)) >> 8;
}
/*********************只不包含浮点运算********************/
//如果使用原始的转换出来的话，会出现偏色问题，下面在原始的基础上又做了微调，可以正常显示。
#define RGB2Y(r,g,b) \((unsigned char)((66 * r + 129 * g + 25 * b + 128) >> 8) + 16)
//  (((r << 6) + (r << 3) + (r << 2) + r + (g << 7) + (g << 4) + (g << 2) + (g << 1) + (b << 4) + (b << 3) + (b << 2) + b) >> 8)#define RGB2U(r,g,b) \((unsigned char)((-38 * r - 74 * g + 112 * b + 128) >> 8) + 128)
//  ((-((r << 5) + (r << 2) + (r << 1)) - ((g << 6) + (g << 3) + (g << 1)) + ((b << 6) + (b << 5) + (b << 4))) >> 8)#define RGB2V(r,g,b) \((unsigned char)((112 * r - 94 * g - 18 * b + 128) >> 8) + 128)
//  (((r << 7) + (r << 4) + (r << 3) + (r << 2) + (r << 1) - ((g << 7) + (g << 2)) - ((b << 4) + (b << 3) + (b << 1))) >> 8)/**********************不包含浮点和乘法运算*****************/
#define RGB2Y_SHIFT(r,g,b) \((unsigned char)((((r << 6) + (r << 3) + (r << 2) + r + (g << 7) + (g << 4) + (g << 2) + (g << 1) + (b << 4) + (b << 3) + (b << 2) + b) + 128) >> 8) + 16)#define RGB2U_SHIFT(r,g,b) \((unsigned char)(((-((r << 5) + (r << 2) + (r << 1)) - ((g << 6) + (g << 3) + (g << 1)) + ((b << 6) + (b << 5) + (b << 4))) + 128) >> 8) + 128)#define RGB2V_SHIFT(r,g,b) \((unsigned char)(((r << 7) + (r << 4) + (r << 3) + (r << 2) + (r << 1) - ((g << 7) + (g << 2)) - ((b << 4) + (b << 3) + (b << 1)) + 128) >> 8) + 128)/*************************包含浮点和乘法运算，没有优化**************/#define RGB2Y_FLOAT(r,g,b) \((unsigned char)(0.299*r + 0.587*g + 0.114*b))#define RGB2U_FLOAT(r,g,b) \((unsigned char)(-0.147*r - 0.289*g + 0.436*b))#define RGB2V_FLOAT(r,g,b) \((unsigned char)(0.615*r - 0.515*g - 0.100*b))void rgb2yuv420_multip(unsigned char *rgb_buf, unsigned char *yuv_buf, int width, int heigh) {if (!rgb_buf || !yuv_buf || !width || !heigh) {printf("invalid param\n");return;}unsigned char *y = yuv_buf;unsigned char *u = yuv_buf + width*heigh;unsigned char *v = u + width*heigh/4;for (int r = 0; r < heigh; r++ ) {for (int c = 0; c < width; c++) {int index = r*width + c;*y++ = RGB2Y(rgb_buf[index * 3 + 2], rgb_buf[index * 3 + 1], rgb_buf[index * 3]);if((r%2==0)&&(c%2 ==0)) //偶数行，偶数列获取U*u++ = RGB2U(rgb_buf[index * 3 + 2], rgb_buf[index * 3 + 1], rgb_buf[index * 3]);else if(c%2 == 0) //奇数行，偶数列获取V*v++ = RGB2V(rgb_buf[index * 3 + 2], rgb_buf[index * 3 + 1], rgb_buf[index * 3]);}}}void rgb2yuv420_shift(unsigned char *rgb_buf, unsigned char *yuv_buf, int width, int heigh) {if (!rgb_buf || !yuv_buf || !width || !heigh) {printf("invalid param\n");return;}unsigned char *y = yuv_buf;unsigned char *u = yuv_buf + width*heigh;unsigned char *v = u + width*heigh/4;for (int r = 0; r < heigh; r++ ) {for (int c = 0; c < width; c++) {int index = r*width + c;*y++ = RGB2Y(rgb_buf[index * 3 + 2], rgb_buf[index * 3 + 1], rgb_buf[index * 3]);if((r%2==0)&&(c%2 ==0)) //偶数行，偶数列获取U*u++ = RGB2U(rgb_buf[index * 3 + 2], rgb_buf[index * 3 + 1], rgb_buf[index * 3]);else if(c%2 == 0) //奇数行，偶数列获取V*v++ = RGB2V(rgb_buf[index * 3 + 2], rgb_buf[index * 3 + 1], rgb_buf[index * 3]);}}}void rgb2yuv420_float(unsigned char *rgb_buf, unsigned char *yuv_buf, int width, int heigh) {if (!rgb_buf || !yuv_buf || !width || !heigh) {printf("invalid param\n");return;}unsigned char *y = yuv_buf;unsigned char *u = yuv_buf + width*heigh;unsigned char *v = u + width*heigh/4;for (int r = 0; r < heigh; r++ ) {for (int c = 0; c < width; c++) {int index = r*width + c;*y++ = RGB2Y_FLOAT(rgb_buf[index * 3 + 2], rgb_buf[index * 3 + 1], rgb_buf[index * 3]);if((r%2==0)&&(c%2 ==0)) //偶数行，偶数列获取U*u++ = RGB2U_FLOAT(rgb_buf[index * 3 + 2], rgb_buf[index * 3 + 1], rgb_buf[index * 3]);else if(c%2 == 0) //奇数行，偶数列获取V*v++ = RGB2V_FLOAT(rgb_buf[index * 3 + 2], rgb_buf[index * 3 + 1], rgb_buf[index * 3]);}}}int main(int arc, const char ** argv)
{int ret = 0;FILE *pFile = NULL,*pFile_out = NULL;int size = 0;int width = 0, heigh = 0;char path[] = "D:\\vs2015_project\\yuv22rgb\\girl320x240.yuv";unsigned char * rgb_buf = NULL;unsigned char *yuv_buf = nullptr;cout << "arc:" << arc << endl;if (arc == 1) {cout << "please input parameters!" << endl;return -1;}sscanf(argv[2],"%d", &width);sscanf(argv[3],"%d", &heigh);printf("file name:%s,width:%d,heigh:%d\n", argv[1], width, heigh);//cout << "file name:" << argv[1] << endl;pFile = fopen(argv[1], "rb");if (NULL == pFile) {cout << "open file failed!" << endl;}else {fseek(pFile, 0, SEEK_END);size = ftell(pFile);fseek(pFile, 0, SEEK_SET);cout << "file size is " << size << endl;}rgb_buf = (unsigned char *)malloc(size);if (NULL == rgb_buf) {cout << "malloc buffer failed, return" << endl;fclose(pFile);return -1;}int r_size = fread(rgb_buf, 1, size, pFile);if (r_size != size) {cout << "read error" << endl;ret = -1;goto release;}int yuv_size = width * heigh * 1.5;yuv_buf = (unsigned char*)malloc(yuv_size);if (NULL == yuv_buf)goto release;LARGE_INTEGER t1, t2,t3,t4;LARGE_INTEGER f;QueryPerformanceFrequency(&f); //得到CPU频率QueryPerformanceCounter(&t1); //得到开始时间rgb2yuv420_multip(rgb_buf, yuv_buf, width, heigh);QueryPerformanceCounter(&t2); //得到优化了浮点运算的计算时间double total1_time = (t2.QuadPart - t1.QuadPart) / (double)f.QuadPart; //单位秒rgb2yuv420_shift(rgb_buf, yuv_buf, width, heigh);QueryPerformanceCounter(&t3); //得到优化浮点和乘法运算之后结束时间double total2_time = (t3.QuadPart - t2.QuadPart) / (double)f.QuadPart; //单位秒rgb2yuv420_float(rgb_buf, yuv_buf, width, heigh);QueryPerformanceCounter(&t4); //原始没有优化的图像double total3_time_float = (t4.QuadPart - t3.QuadPart) / (double)f.QuadPart; //单位秒printf("shift_time:%f s,multip_time:%f s,time_float:%f s\n", total2_time,total1_time,total3_time_float);pFile_out = fopen(path, "wb+");if (pFile_out == nullptr) {printf("null fd\n");goto release;}printf("yuv_size:%d\n", yuv_size);int w_count = fwrite(yuv_buf, 1, yuv_size, pFile_out);printf("write_count:%d\n", w_count);fclose(pFile_out);release:if (rgb_buf) {free(rgb_buf);rgb_buf = nullptr;}if (yuv_buf) {free(yuv_buf);yuv_buf = nullptr;}fclose(pFile);return 0;
}

运行结果如下所示：

D:\vs2015_project\yuv22rgb\yuv22rgb\Debug>yuv22rgb.exe …\girl320x240.rgb 320 240
arc:4
file name:…\girl320x240.rgb,width:320,heigh:240
file size is 230400
shift_time:0.000367 s,multip_time:0.000366 s,time_float:0.000593 s
yuv_size:115200
write_count:115200

从上面能够发现在同样的运行环境下，处理时间优化前后差异很大，其中优化移位操作，优化的时间不是很大，不知道跑在arm架构上时间会是怎么样的，后面会尝试一下。

优化项	时间(单位S)
没有任何优化	0.000593
优化浮点计算	0.000367
优化浮点和乘法计算	0.000366

2.CPU硬件信息
3.图片对比,下图左边是原始的图片，右边是转换之后的YUV，可以发现转换之后的图片有点偏红，不过色调基本一致。

五、编译过程中遇到的问题

1.读取文件识别
打开文件时应该加上“rb"模式
2.写入文件有问题(写入文件的大小和预期不一致)
写入文件时应该使用”wb+"
3.图像偏色
uv反色或者公式有问题