ndk实例总结

ndk实例总结：jni实例
ndk实例总结：opencv图像处理
ndk实例总结：安卓Camera与usbCamera原始图像处理
ndk实例总结补充：使用V4L2采集usb图像分析
ndk实例总结：使用fmpeg播放rtsp流
ndk实例总结：基于libuvc的双usbCamera处理
ndk实例总结补充：使用libuvc采集usb图像分析
ndk实例总结：jni日志存储

前言

在Android上进行usb camera相关的开发，离不开两种方式，一种基于v4l2接口，我在ndk实例总结补充：使用V4L2采集usb图像分析
中有过分析，另一种基于libuvc，github上有一个开源项目UVCCamera，但是封装的非常复杂，很难在其基础上进行定制开发，而且不适合用来学习

所以这个项目基于libuvc原本的api进行定制开发，实现了双摄像头的预览与拍照，以及更改摄像头参数等相关功能

项目当中用到了从UVCCamera项目里拆出来的一些包装类，比如USBMonitor、USBVendorId、DeviceFilter等，so库使用了UVCCamera已经编译好的so库，相比libuvc源码，它在Android上做了很多优化

项目构架

CMakeLists相关配置与ndk实例总结系列的其他项目类似，这里不再介绍了

流程分析

libuvc与v4l2最大的不同就是libuvc是基于usb设备的vid、pid、fd、busNum、devNum之类的信息来操控摄像头，而v4l2是通过/dev/video*的设备号来操控

因此大致流程就是：获取usbDevice相关信息，将usb信息传递到jni中使用libuvc api打开摄像头，以及在jni中获取摄像头原始图像并处理，最后绘制到SurfaceView上

初始化

@Override
public void onCreate(Bundle savedInstanceState) {mManager = UsbCameraManager.getInstance();mManager.init(this.getApplicationContext(), this);// mManager.setStreamMode(UsbCameraManager.ASYNC_CALLBACK);mManager.setStreamMode(UsbCameraManager.SYNC_POLLING);
}@Override
protected void onStart() {super.onStart();// 注册BroadcastReceiver监视USB事件mManager.register();
}

public void init(Context context, final OnUsbConnectListener listener) {this.mContext = context;Arrays.fill(mDeviceArr, null);Arrays.fill(mCtrlBlockArr, null);Arrays.fill(mCamViewArr, null);Arrays.fill(mIsOpenArr, null);mUsbMonitor = new USBMonitor(context, new USBMonitor.OnDeviceConnectListener() {@Overridepublic void onAttach(UsbDevice device) {Log.i(TAG, "onAttach: ");if (listener != null) {listener.onAttachDev(device);}}@Overridepublic void onDetach(UsbDevice device) {Log.i(TAG, "onDetach: ");if (listener != null) {listener.onDetachDev(device);}}@Overridepublic void onConnect(UsbDevice device, USBMonitor.UsbControlBlock ctrlBlock, boolean createNew) {Log.i(TAG, "onConnect: ");if (listener != null) {listener.onConnectDev(device, ctrlBlock, createNew);}}@Overridepublic void onDisconnect(UsbDevice device, USBMonitor.UsbControlBlock ctrlBlock) {Log.i(TAG, "onDisconnect: ");if (listener != null) {listener.onDisconnectDev(device, ctrlBlock);}}@Overridepublic void onCancel(UsbDevice device) {Log.i(TAG, "onCancel: ");}});
}

init方法中初始化USBMonitor类，并且实例化了几个回调状态，分别对应摄像头的插拔状态，在调用了mManager.register方法后系统广播就会回调相关的USB状态

@IntDef({ASYNC_CALLBACK, SYNC_POLLING})
@interface StreamMode {}public void setStreamMode(@StreamMode int streamMode) {this.streamMode = streamMode;
}

streamMode代表获取摄像头原始帧的两种方式，一种是异步回调方式（传递callback函数到libuvc开启流的函数中，每帧图像就会通过callback函数回调），另一种是主动轮寻方式（主动调用libuvc的获取帧函数，调用一次获取一帧）

根据项目自己选择使用哪种方式，比如需要预览就使用异步回调方式，只拍照不预览就可以选择主动轮寻方式

摄像头打开与关闭

摄像头打开与关闭的逻辑在USB回调事件中，在onAttach中获取权限，onConnect中打开摄像头，onDetach或onDisconnect中关闭摄像头

@Override
public void onAttachDev(UsbDevice usbDevice) {Log.i(TAG, "onAttachDev: USB_DEVICE_ATTACHED " + usbDevice.toString());runOnUiThread(() -> Toast.makeText(this, "USB_DEVICE_ATTACHED", Toast.LENGTH_SHORT).show());if (!mManager.isRequestPermission(usbDevice)) {mManager.requestPermission(usbDevice);}
}@Override
public void onDetachDev(UsbDevice usbDevice) {Log.i(TAG, "onDetachDev: USB_DEVICE_DETACHED " + usbDevice.toString());runOnUiThread(() -> Toast.makeText(this, "USB_DEVICE_DETACHED", Toast.LENGTH_SHORT).show());mManager.closeCamera(usbDevice);
}@Override
public void onConnectDev(UsbDevice usbDevice, USBMonitor.UsbControlBlock usbControlBlock, boolean createNew) {Log.i(TAG, "onConnectDev: USB_DEVICE_CONNECT " + usbDevice.toString());if (!mManager.isCameraOpened(usbControlBlock)) {boolean ok = false;if (mManager.findCameraViewIndex(cameraPreviewOne) < 0) {ok = mManager.openCamera(usbDevice, usbControlBlock, cameraPreviewOne);} else if (mManager.findCameraViewIndex(cameraPreviewTwo) < 0) {ok = mManager.openCamera(usbDevice, usbControlBlock, cameraPreviewTwo);}if (ok) setParam(usbDevice);// 如果有摄像头打开失败的话可以调用摄像头复位接口，比如MTK平台的切换OTG接口if (mManager.isCameraOpenFail()) resetUsb();}
}@Override
public void onDisconnectDev(UsbDevice usbDevice, USBMonitor.UsbControlBlock usbControlBlock) {Log.i(TAG, "onDisconnectDev: USB_DEVICE_DISCONNECT " + usbDevice.toString());mManager.closeCamera(usbDevice);
}

获取权限与关闭摄像头逻辑都比较简单，这里主要介绍下打开摄像头，首先需要判断摄像头是否已打开，isCameraOpened会判断UsbControlBlock对象是否存在，不存在则表示未打开

public boolean isCameraOpened(USBMonitor.UsbControlBlock ctrlBlock) {return findUsbControlBlockIndex(ctrlBlock) >= 0;
}public int findUsbControlBlockIndex(USBMonitor.UsbControlBlock ctrlBlock) {for (int i = 0; i < mCtrlBlockArr.length; ++i) {if (ctrlBlock.equals(mCtrlBlockArr[i])) {return i;}}return -1;
}

然后看一下打开摄像头相关代码，首先判断UsbDevice对象是否存在（该对象在获取到权限时会保存），然后打开摄像头并保存UsbControlBlock对象，最后通过jni调用libuvc的打开摄像头相关代码

public boolean openCamera(UsbDevice dev, USBMonitor.UsbControlBlock ctrlBlock, CameraViewInterface cameraView) {int index = findUsbDeviceIndex(dev);if (index >= 0) {Log.i(TAG, "openCamera: index " + index);Log.i(TAG, "openCamera: cameraView " + cameraView);int result = openCamera(index, ctrlBlock);if (result == 0) {mCtrlBlockArr[index] = ctrlBlock;mIsOpenArr[index] = true;if (cameraView != null) {cameraView.openCamera(index);mCamViewArr[index] = cameraView;}return true;} else {mIsOpenArr[index] = false;Log.e(TAG, "openCamera: error " + result);}} else {Log.e(TAG, "openCamera: usbDevice not found");}return false;
}public int openCamera(int index, USBMonitor.UsbControlBlock ctrlBlock) {Log.i(TAG, "openCamera: ");Log.i(TAG, "getVendorId: " + ctrlBlock.getVendorId());Log.i(TAG, "getProductId: " + ctrlBlock.getProductId());Log.i(TAG, "getFileDescriptor: " + ctrlBlock.getFileDescriptor());Log.i(TAG, "getBusNum: " + ctrlBlock.getBusNum());Log.i(TAG, "getDevNum: " + ctrlBlock.getDevNum());Log.i(TAG, "getUSBFSName: " + ctrlBlock.getUSBFSName());UsbCameraLib.setStreamMode(streamMode);return UsbCameraLib.connect(index, ctrlBlock.getVendorId(), ctrlBlock.getProductId(),ctrlBlock.getFileDescriptor(), ctrlBlock.getBusNum(), ctrlBlock.getDevNum(),ctrlBlock.getUSBFSName());
}public int findUsbDeviceIndex(UsbDevice dev) {for (int i = 0; i < mDeviceArr.length; ++i) {UsbDevice usbDevice = mDeviceArr[i];if (dev.equals(usbDevice)) {return i;}}return -1;
}public int findCameraViewIndex(CameraViewInterface cameraView) {for (int i = 0; i < mCamViewArr.length; ++i) {if (cameraView.equals(mCamViewArr[i])) {return i;}}return -1;
}

摄像头打开成功后可以设置摄像头参数参数，可以设置曝光方式（自动或手动），曝光时长（手动曝光可用），增益（gain）和亮度

private void setParam(UsbDevice usbDevice) {mManager.setParam(usbDevice, 0, 0.0157f, 0, 50);// mManager.setParam(usbDevice, 1, 0.015f, 15, 50);
}

如果在openCamera中传入CameraViewInterface的话则表示需要预览图像，CameraPreview这个SurfaceView实现了这个接口，会保存摄像头的index值用来具体对应不同的摄像头，以及控制打开摄像头的标志位

public interface CameraViewInterface {void openCamera(int index);void closeCamera();
}public class CameraPreview extends SurfaceView implements SurfaceHolder.Callback, Runnable, CameraViewInterface {...@Overridepublic void openCamera(int index) {Log.i(TAG, "openCamera: " + index);isOpen = true;this.index = index;}@Overridepublic void closeCamera() {Log.i(TAG, "closeCamera: " + index);isOpen = false;this.index = -1;}
}

预览图像

使用SurfaceView进行绘制，在while循环中通过jni将原始帧数据放入bitmap中

@Override
public void run() {while (!Thread.interrupted()) {if (isOpen && index >= 0) {UsbCameraLib.pixeltobmp(index, bmp);}Canvas canvas = getHolder().lockCanvas();if (canvas != null) {canvas.drawBitmap(bmp, mSrcRect, mDstRect, null);getHolder().unlockCanvasAndPost(canvas);}frame++;}
}@Override
public void surfaceCreated(SurfaceHolder holder) {Log.i(TAG, "surfaceCreated");if (bmp == null) {bmp = Bitmap.createBitmap(IMG_WIDTH, IMG_HEIGHT, Bitmap.Config.RGB_565);}mSrcRect = new Rect(0, 0, bmp.getWidth(), bmp.getHeight());mDstRect = new Rect(0, 0, getWidth(), getHeight());mainLoop = new Thread(this);mainLoop.start();
}@Override
public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {Log.i(TAG, "surfaceChanged");
}@Override
public void surfaceDestroyed(SurfaceHolder holder) {Log.i(TAG, "surfaceDestroyed");closeCamera();mainLoop.interrupt();if (bmp != null) {bmp.recycle();}
}

看一下jni中的pixel_to_bmp函数，会将rgb565的图片像素数据直接memcpy到传递下来的bitmap对象中

void pixel_to_bmp(JNIEnv *env, jclass thiz, int index, jobject bitmap) {int ret;uvc_frame_t *rgb565 = uvc_allocate_frame(IMG_WIDTH * IMG_HEIGHT * 2);ret = get_frame(rgb565, index);if (ret) {LOGE("pixel_to_bmp get_frame error %d", ret);uvc_free_frame(rgb565);return;}AndroidBitmapInfo info;void *pixels;int i;int *colors;if ((ret = AndroidBitmap_getInfo(env, bitmap, &info)) < 0) {LOGE("pixel_to_bmp AndroidBitmap_getInfo() failed ! error=%d", ret);uvc_free_frame(rgb565);return;}int width = info.width;int height = info.height;if (info.format != ANDROID_BITMAP_FORMAT_RGB_565) {LOGE("pixel_to_bmp Bitmap format is not RGB_565!");return;}if ((ret = AndroidBitmap_lockPixels(env, bitmap, &pixels)) < 0) {LOGE("pixel_to_bmp AndroidBitmap_lockPixels() failed ! error=%d", ret);uvc_free_frame(rgb565);return;}colors = (int *) pixels;memcpy(colors, rgb565->data, rgb565->data_bytes);uvc_free_frame(rgb565);AndroidBitmap_unlockPixels(env, bitmap);
}

最后完整代码可以参考demo

ndk开发基础学习系列

JNI和NDK编程（一）JNI的开发流程
JNI和NDK编程（二）NDK的开发流程
JNI和NDK编程（三）JNI的数据类型和类型签名
JNI和NDK编程（四）JNI调用Java方法的流程

完整demo

https://github.com/GavinAndre/AndroidJNIDemo