音视频系列

如何进行视频的采集和预览

视频的采集需要用到Camera这个API，谷歌在5.0引入了camrea2，为了适配所有的机型，我将分别介绍camera和camera2
预览可以使用SurfaceView和TextureView

<uses-permission android:name="android.permission.CAMERA" />

采集

camera的基本使用

1.打开摄像头

mCamera=Camera.open(mCameraId);

mCameraId为int型，1代表前置摄像头，0代表后置摄像头

2.设置预览的媒介

mCamera.setPreviewDisplay(mHolder);
mCamera.setPreviewTexture(mTexture); 

前者是SurfaceView预览，后者是TextureView预览

3.设置参数

Camera.Parameters parameters = mCamera.getParameters();
parameters.setPreviewFormat(ImageFormat.NV21); //预览数据格式
parameters.setPictureFormat(ImageFormat.JPEG);  //设置拍照图片格式
parameters.setPreviewSize(mSize.width, mSize.height);   //预览视频尺寸
parameters.setPictureSize(nSize.width, nSize.height); //拍照图片尺寸 

4.设置预览方向和拍摄图片的方向

 //设置拍照后图片的方向，否则方向不对if (mCameraId == 0) {parameters.setRotation(90); //后置 } else { parameters.setRotation(270); //前置 } mCamera.setParameters(parameters); mCamera.setDisplayOrientation(90); //预览角度默认0度，手机左侧为0 

5.开启预览

 mCamera.setPreviewCallback(this);  //接收每一帧的数据mCamera.startPreview();

camera2的基本使用

camera2相较于camera发生了很大的变化

先上两张网上找来的图

camera2架构图.png

camera2流程示意图.png

图一说明
Android Device相当于我们的app
Camera Device相当于手机上的相机
app想要使用相机，就需要在app和相机之间建立一个连接
连接建立完成之后app就可以向相机发起数据请求
相机响应请求并向app返回数据
app得到数据之后可以通过各种surface处理数据

图二说明
图二介绍了连接过程中用到的关键类

CameraManager
负责管理所有的摄像头，如打开相机

CameraDevice.StateCallback
相机设备状态的回调，可以判断相机是否打开

CaptureRequest.Builder
通过Builder.build()创建一个CaptureRequest，如预览，拍照，录像等等
通过addTarget(Surface outputTarget)为对应的每一个CaptureRequest提供数据处理的地方，即数据着陆点

CameraCaptureSession.StateCallback
通过 mCameraDevice.createCaptureSession创建一次会话，到了这里app和相机才算了建立起了连接

camera2流程梳理

预览

1.通过CameraManager打开相机

mCameraManager = (CameraManager) mContext.getSystemService(Context.CAMERA_SERVICE);
mCameraManager.openCamera(String.valueOf(mCameraId),new CameraStateCallback(), mCameraHandler);

2.在CameraDevice.StateCallback的onOpened回调中拿到相机对象，同时创建一个预览的请求

@Override
public void onOpened(@NonNull CameraDevice camera) { mCameraDevice = camera; startPreview(); } CaptureRequest.Builder builder = mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW); Surface surface = getSurface(); builder.addTarget(surface); mCaptureRequest = builder.build(); 

3.通过相机对象建立一个连接

mCameraDevice.createCaptureSession(Arrays.asList(surface, mPictureReader.getSurface()), new CameraSessionCallback(), mCameraHandler); 

4.在连接建立成功的回调中发送之前创建的预览这个请求

public void onConfigured(@NonNull CameraCaptureSession session) { try { mCameraCaptureSession = session; //预览 mCameraCaptureSession.setRepeatingRequest(mCaptureRequest, null, null); } catch (CameraAccessException e) { e.printStackTrace(); } } 

至此整个预览就完成了

拍照
拍照相当于一个CaptureRequest，如果我们已经完成了预览的流程，对于拍照只需要新创建一个请求，然后通过之前创建的Session发送这个请求即可。
在camera2中一次会话可以包含多个请求，因此我们只需要创建一次会话即可

public void takePicture(String path) { mPicturePath = path; try { CaptureRequest.Builder builder = mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_STILL_CAPTURE); builder.addTarget(mPictureReader.getSurface()); //拍照 mCameraCaptureSession.capture(builder.build(), null, null); } catch (CameraAccessException e) { e.printStackTrace(); } } 

mPictureReader的写法

//2代表ImageReader中最多可以获取两帧图像流
mPictureReader = ImageReader.newInstance(mCaptureSize.getWidth(), mCaptureSize.getHeight(),ImageFormat.JPEG, 2);
mPictureReader.setOnImageAvailableListener(new ImageReader.OnImageAvailableListener() {@Override public void onImageAvailable(ImageReader reader) { //将这帧数据转成字节数组，类似于Camera1的PreviewCallback回调的预览帧数据 savePicture(reader); } }, mCameraHandler); 

由于我们设置的图片是JPEG格式，则在onImageAvailable回调中，直接可以将字节数组保存为jpg格式的图片

 private void savePicture(ImageReader reader) { Image image = reader.acquireLatestImage(); ByteBuffer buffer = image.getPlanes()[0].getBuffer(); byte[] data = new byte[buffer.remaining()]; buffer.get(data); try { FileOutputStream fos = new FileOutputStream(mPicturePath); fos.write(data); fos.close(); Toast.makeText(mContext,"图片保存至:"+mPicturePath,Toast.LENGTH_SHORT).show(); } catch (FileNotFoundException e) { e.printStackTrace(); Log.d(TAG, "Camera2 FileNotFoundException"); } catch (IOException e) { e.printStackTrace(); Log.d(TAG, "Camera2 IOException"); } finally { image.close(); } } 

摄像
摄像也相当于一个请求，我们同样可以创建一个摄像的请求，然后通过之前创建的session将这个请求发送出去
摄像和预览一样都是通过在CameraCaptureSession.StateCallback的回调中调用setRepeatingRequest方法

两者的区别
预览是将数据展示在SurfaceView或者TextureView上
摄像是将数据保存成一个视频文件

摄像数据的保存
通过ImageReader拿到原始的NV21数据

ImageReader.newInstance(width,height,ImageFormat.NV21, 1); 

注意Camera2已经不支持NV21格式了，通过源码可以看出当使用该格式时，会抛异常

 protected ImageReader(int width, int height, int format, int maxImages, long usage) { mWidth = width; mHeight = height; mFormat = format; mMaxImages = maxImages; if (width < 1 || height < 1) { throw new IllegalArgumentException( "The image dimensions must be positive"); } if (mMaxImages < 1) { throw new IllegalArgumentException( "Maximum outstanding image count must be at least 1"); } if (format == ImageFormat.NV21) { throw new IllegalArgumentException( "NV21 format is not supported"); } mNumPlanes = ImageUtils.getNumPlanesForFormat(mFormat); nativeInit(new WeakReference<>(this), width, height, format, maxImages, usage); mSurface = nativeGetSurface(); mIsReaderValid = true; // Estimate the native buffer allocation size and register it so it gets accounted for // during GC. Note that this doesn't include the buffers required by the buffer queue // itself and the buffers requested by the producer. // Only include memory for 1 buffer, since actually accounting for the memory used is // complex, and 1 buffer is enough for the VM to treat the ImageReader as being of some // size. mEstimatedNativeAllocBytes = ImageUtils.getEstimatedNativeAllocBytes( width, height, format, /*buffer count*/ 1); VMRuntime.getRuntime().registerNativeAllocation(mEstimatedNativeAllocBytes); } 

解决办法，改为YUV_420_888格式

mPreviewReader = ImageReader.newInstance(1280, 720,ImageFormat.YUV_420_888, 1);

然后将YUV_420_888格式转化为NV21即可，具体转换方式可以百度
参考链接

通过MediaRecorder直接存储为经过编码压缩的文件

mMediaRecorder.setOutputFormat(MediaRecorder.OutputFormat.MPEG_4); 

两者区别
前者更灵活，拿到原始数据之后可以做一些自己的处理
后者更简单，无需我们在编码和压缩了

封装

前面已经介绍了camera和camera2的用法，通过代码的练习，相信大家已经了解了两种camera的基本用法了

那么问题来了，我们怎样用这两种camera了
由于camera2是5.0中的api，为了兼容旧版手机，我们又不得不继续使用camera，但两者的用法却完全不一样
为此我封装了一个SofarCamera类，为camera和camera2的提供了统一的调用方法，只需要一个参数的区分，用户就可以自由切换要使用的是旧的camera还是5.0的camera2

同时在封装过程中，我将数据和UI界面分离开，读者可以自己设计UI界面，用的时候只需要传递SurfaceView的holder或者TextureView的texture即可

protected SurfaceHolder mHolder;  //SurfaceView预览
protected SurfaceTexture mTexture;  //TextureView预览 

对外统一调用类SofarCamera

public class SofarCamera { public static final int CAMERA_FRONT=1; //前置摄像头 public static final int CAMERA_BACK=0; //后置摄像头 public static final int CAMERA1=1; //旧api public static final int CAMERA2=2; //新api private Context context; private int cameraId; private int cameraApi; private SurfaceHolder holder; private SurfaceTexture texture; private BaseCamera baseCamera; private SofarCamera(Builder builder){ this.context=builder.context; this.cameraId=builder.cameraId; this.cameraApi=builder.cameraApi; this.holder=builder.holder; this.texture=builder.texture; initCamera(); } private void initCamera(){ if(cameraApi==CAMERA1){ baseCamera=new Camera1(); }else if(cameraApi==CAMERA2){ baseCamera=new Camera2(); } baseCamera.setContext(context); baseCamera.setCameraId(cameraId); baseCamera.setDisplay(holder); baseCamera.setDisplay(texture); } public void openCamera(){ baseCamera.openCamera(); } public void destroyCamera(){ baseCamera.destroyCamera(); } public void switchCamera(){ if(cameraId==CAMERA_FRONT){ cameraId=CAMERA_BACK; }else { cameraId=CAMERA_FRONT; } destroyCamera(); initCamera(); openCamera(); } public void takePicture(String path){ baseCamera.takePicture(path); } public Builder newBuilder() { return new Builder(this); } public static final class Builder{ private Context context; private int cameraId; private int cameraApi; private SurfaceHolder holder; private SurfaceTexture texture; public Builder(){ } public Builder(SofarCamera camera){ this.cameraId=camera.cameraId; this.cameraApi=camera.cameraApi; this.holder=camera.holder; this.texture=camera.texture; } public Builder context(Context context){ this.context=context; return this; } public Builder cameraId(int cameraId){ this.cameraId=cameraId; return this; } public Builder cameraApi(@CameraApi int cameraApi){ this.cameraApi=cameraApi; return this; } public Builder holder(SurfaceHolder holder){ this.holder=holder; return this; } public Builder texture(SurfaceTexture texture){ this.texture=texture; return this; } public SofarCamera build() { return new SofarCamera(this); } } @IntDef({CAMERA1,CAMERA2}) @Retention(RetentionPolicy.SOURCE) public @interface CameraApi{ } } 

Camera的抽象
对于Camera1和Camera2只需要继承该类，并实现必要的方法即可

public abstract class BaseCamera { private static final String TAG = "BaseCamera"; protected Context mContext; protected int mCameraId = 1; //1前置 0后置 protected SurfaceHolder mHolder; //SurfaceView预览 protected SurfaceTexture mTexture; //TextureView预览 protected String mPicturePath; //拍照后的图片存储路径 //SurfaceView预览 public void setDisplay(SurfaceHolder holder) { mHolder = holder; } //TextureView预览 public void setDisplay(SurfaceTexture texture) { mTexture = texture; } //设置前置或后置摄像头 public void setCameraId(int cameraId) { if (cameraId != 1 && cameraId != 0) { Log.d(TAG, "error cameraId:" + cameraId + " BaseCamera cameraId only support 0 or 1 "); return; } mCameraId = cameraId; } public void setContext(Context context) { mContext = context; } public abstract void openCamera(); public abstract void destroyCamera(); public abstract void takePicture(String path); } 

调用方法

使用camera

mSofarCamera = new SofarCamera.Builder().context(this).cameraApi(SofarCamera.CAMERA1).cameraId(SofarCamera.CAMERA_BACK).holder(holder).build();

使用camera2

mSofarCamera = new SofarCamera.Builder().context(this).cameraApi(SofarCamera.CAMERA2).cameraId(SofarCamera.CAMERA_BACK).holder(holder).build();

最后的话

由于篇幅有限，Camera1和Camera2的代码我就不贴了
读者可以阅读前面的使用介绍或者一些其他资料，自己来实现它

我的实现已经放置在github上
https://github.com/hustersf/SofarMusic

Camera的封装放置在libplayer下的video包下
方法调用放置在app/demo/media/video下