I always love to play with random geeky stuff, Recently thought of diving into video encoding stuff in React Native. Guess what, I loved it and did some amazing stuff during the weekend.

我一直喜欢玩随意的怪异事物，最近想到在React Native中涉足视频编码领域。 猜猜我喜欢它，并且在周末做了一些很棒的事情。

It might seem a bit advanced as you need some prior knowledge on encoding terminology & FFMPEG etc. But trust me it will be interesting.

由于您需要一些有关编码术语和FFMPEG等的先验知识，因此它似乎有点高级。但是请相信我，这将很有趣 。

我计划了几件事 (I have planned few things)

1. A video recording component using RNCamera and few configurations to set — Bitrate, Aspect ratio, Permissions etc.

   一个使用RNCamera的视频录制组件，并需要设置一些配置-比特率，宽高比，权限等。

2. Flexible to handle multiple clips in single session.

   灵活地在单个会话中处理多个剪辑。

3. A song picker to play a song in background of the recording.

   在录音背景中播放歌曲的拾音器。

4. Once recorded, Merge all the recorded clips.

   录制后，合并所有录制的剪辑。

5. Option to delete the clips.

   选择删除剪辑。

6. Option to trim the video by showing video frames & time slider.

   通过显示视频帧和时间滑块来修剪视频的选项。

7. Final Encoding — Trimmed video + Song + Watermark + Optimised output.

   最终编码-修整后的视频+歌曲+水印+优化的输出。

So let’s go through one by one.

因此，让我们一一讲解。

RN Camera的视频录制组件 (Video recording component with RN Camera)

Have used RN Camera — V3.31.1 https://github.com/react-native-community/react-native-camera

已使用RN相机-V3.31.1 https://github.com/react-native-community/react-native-camera

As you can observe the above code snippet.

如您所见，上面的代码段。

1. Zoom controls缩放控制
2. FlashMode闪光模式
3. Progress tracker进度追踪器
4. Event handlers for Recording, Camera state & Few AR stuff.记录，相机状态和少量AR内容的事件处理程序。
5. Native Permissions.本机权限。

在单个会话中处理多个短片。 (Handle multiple short clips in single session.)

On recording start we can invoke this library method recordAsync with few config props, Such as (Quality, Bitrate etc). As this is a lovely promise it always returns whenever the recording stops. Post that we can find whether it is an end of recording or paused depends on the progress & timer.

录制开始时，我们可以使用一些配置道具(例如(质量，比特率等))调用此库方法recordAsync 。 因为这是一个可爱的承诺，所以只要录音停止，它总是会返回。 我们可以确定是结束录制还是暂停录制的帖子取决于进度和计时器。

Accordingly we can store the clips & modify the state.

因此，我们可以存储剪辑并修改状态。

选歌者 (A song picker)

Have used RNGetMusicFileshttps://www.npmjs.com/package/react-native-get-music-files

使用了RNGetMusicFiles https://www.npmjs.com/package/react-native-get-music-files

There are few issues in this package and also will give lot of conflicts when used with FFMPEG. So have modified the codebase and converted it to an internal dependency via vendor. (Converting external libraries to internal is also dam interesting stuff — Will explain in a separate blog)

该软件包中的问题很少，与FFMPEG一起使用时也会产生很多冲突。 因此，已经修改了代码库，并通过供应商将其转换为内部依赖项。 (将外部库转换为内部库也是非常有趣的事情-将在单独的博客中进行解释)

Once linked as a local (Internal) dependency. Have removed some unused Android package dependencies. Have also raised PR’s for some common fixes.

一旦链接为本地(内部)依赖项 。 删除了一些未使用的Android软件包依赖项。 还提高了一些常见修复程序的PR。

Coming back to Implementation.

回到实施。

Once we get the list of songs. We can display them accordingly and can handle play pause etc. Once the song is selected here’s how I manage state.

一旦我们获得了歌曲列表。 我们可以相应地显示它们，并可以处理播放暂停等。一旦选择了歌曲，这就是我如何管理状态。

处理录制的剪辑。 (Handling the recorded clips.)

Two ways

两种方式

1. Merging all recorded clips合并所有录制的剪辑
2. User can delete the recent clip following till the beginning.用户可以删除最近的剪辑，直到开始。

删除最近的 (Deleting Recent)

As we know, That we are tracking all the clips in an Array, So this should be relatively easily for us to handle. Just keep popping up and recalculating the timer & progress.

众所周知，我们正在跟踪数组中的所有剪辑，因此，这对于我们来说应该相对容易些。 只是不断弹出并重新计算计时器和进度。

Till the last one.

直到最后一个。

Final reset. So this means user have deleted all the recorded clips.

最终重置。 因此，这意味着用户已删除所有记录的剪辑。

合并中 (Merging)

I am using RNVideoEditor to merge files, Which internally uses MP4parser, Which is also great, but I would prefer FFMPEG, As of now living with this. But have some plans

我正在使用RNVideoEditor合并文件，该文件在内部使用MP4parser，这也很棒，但是我更喜欢FFMPEG，截至目前为止 。 但是有一些计划

To migrate this into local dependency and change the implementation to FFMPEG.orJust remove this dependency and encoding on the flow as we will be discussing in our next steps.

要将其迁移到本地依赖项并将实现更改为FFMPEG，或者仅删除此依赖项并在流上进行编码，这将在下一步中进行讨论。

As per the above snippet, We can easily see that we need merging when there are multiple clips and also merging is relatively clean with RNVideoEditor. But there is a catch. If we try to record clips mix of Front camera and Back camera, Then either it fails sometimes or the clips will be rotated. Which is an issue to be taken care either by disable mix camera recordings or use the native encoding techniques to process the clips.

根据上面的代码片段，我们可以很容易地看到，当有多个剪辑时我们需要合并，并且使用RNVideoEditor合并是相对干净的。 但是有一个陷阱 。 如果我们尝试记录前置摄像头和后置摄像头的混合剪辑，则有时可能会失败或剪辑会旋转。 禁用混合摄像机录制或使用本机编码技术来处理剪辑时要特别注意这一问题。

Let’s slowly dive into FFMPEG now

现在让我们慢慢进入FFMPEG

For Video encoding, I am using RNFFmpeg (full-gps) version

对于视频编码，我使用的是RNFFmpeg (完整gps)版本

Gradle file to include the FFMPEG package

Gradle文件包含FFMPEG包

显示“框架和修剪”选择滑块。 (Showing Frames & Trim selection slider.)

Here we need 2 things.

这里我们需要两件事。

1. Getting frames from the recorded video.从录制的视频中获取帧。
2. Trim Selection slider to select the final version.修剪选择滑块以选择最终版本。

Getting Frames

获取框架

Command for generating video thumbs with a scaling of height 90 and width locked with ASR. As you can observe the command, FPS is calculated in a way that it generates 10 preview thumbnails irrespective of the video duration. We just need to show 10 clean thumbnails for trimming the video.

用于生成高度为90且宽度为ASR的视频缩略图的命令。 如您所见，FPS的计算方式是，无论视频持续时间如何，它都会生成10个预览缩略图。 我们只需要显示10个干净的缩略图即可修剪视频。

修剪选择滑块 (Trim selection slider)

Using MultiSlider https://github.com/ptomasroos/react-native-multi-sliderBut, have customised it and converted as a local dependency.

使用MultiSlider https://github.com/ptomasroos/react-native-multi-slider但是，已经对其进行了自定义并将其转换为本地依赖项。

This slider will be a wrapper on top of video frame previews. Which get’s rendered behind.

此滑块将作为视频帧预览的包装。 哪个被渲染了。

Kinda lazy load till the previews get’s generated and rendered.

Kinda延迟加载，直到生成并渲染预览。

Once the thumbnails get’s generated the slider will be on top of it. So that user can slide the section wherever he want, On sliding the video preview will change as we are handling in the handleSliderChange method

生成缩略图后，滑块将位于其顶部。 这样，用户便可以在任意位置滑动该部分。在滑动视频时，视频预览将随着我们在handleSliderChange方法中的处理而改变

This handles the preview of video and also the background song. As per the slider changes.

这可处理视频预览以及背景歌曲。 根据滑块的变化。

最终编码 (Final Encoding)

Trimmed video + Song + Watermark + Optimised output.

修整后的视频+歌曲+水印+优化的输出。

I love this part and the most interesting section. For video editing we need to very careful in encoding. Because it’s not just encoding a video and generating output. It’s always about how optimised the final output is. Because there is no point if the final video is pretty big or pixelated or lost frames.

我喜欢这一部分，也是最有趣的部分。 对于视频编辑，我们需要非常小心地进行编码。 因为它不只是编码视频并生成输出。 总是要考虑最终输出的优化程度。 因为最终的视频是否很大，像素化或丢帧是没有意义的。

So, Coming to our story now.

所以，现在来看我们的故事。

1. We need to trim the video我们需要修剪视频
2. Add song in the background在后台添加歌曲
3. Add watermark加水印

And generate an output which is at-least 5 times smaller without loosing frames or pixels. Believe me I haven’t got a proper slept for at-least 45 hours. But finally cracked it. (As always ;))

并生成至少小 5倍的输出而不会丢失帧或像素。 相信我，至少有45个小时没有适当的睡眠。 但终于破解了。 (一如既往 ;))

Observe carefully, I am using filter_complex for encoding. Providing 3 inputs

仔细观察，我正在使用filter_complex进行编码。 提供3个输入

1. Video (Trimmed based on selection)视频(根据选择修剪)
2. Watermark水印
3. Song (Trimmed based on selection)歌曲(根据选择进行整理)

And then passing them into filter_complex, In this have lined up the steps

然后将它们传递给filter_complex，在此过程中，它们已排好序

1. Scaling the video input to 540x960 (Aspect Ratio — 1080x1920) and intermediate output as Ov将视频输入缩放为540x960(纵横比— 1080x1920)，中间输出为Ov
2. Formatting and Scaling watermark 72x-1 (Locks the Aspect ratio) as Ol格式化和缩放水印72x-1(锁定纵横比)为Ol
3. Setting overlay with a minor animation — Which slides from right side after few seconds设置带有次要动画的叠加层-几秒钟后从右侧滑动
4. Adding watermark text below it在其下方添加水印文本

Once all the filters mapped then specifying a -CRF 28 (Constant Rate Factor) — The range of the quantizer scale is 0–51: where 0 is lossless, 23 is the default, and 51 is worst possible. A lower value is a higher quality

一旦映射了所有滤波器，然后指定-CRF 28 (恒定速率因子)— 量化器比例的范围是0-51：其中0是无损的，默认是23，最坏的可能是51。 值越低质量越高

Encoding audio -c:a AAC (Advanced Audio Coding) and then mapping the inputs accordingly -map 2:a and generating an output video.

编码音频-c：a AAC(高级音频编码)，然后相应地映射输入-map 2：a并生成输出视频。

15秒的最终编码视频仅为 (The final encoded video of 15 seconds is just)

Android — 1.8 MB

Android — 1.8 MB

IOS — 1.6 MB

IOS — 1.6 MB

There is lot to wrap and play with. Once everything is done will add links to the code and a sample App to play with.

有很多东西可以玩。 完成所有操作后，将添加指向代码的链接和一个示例应用程序。

Meanwhile, Give a try — It will be fun.

同时，尝试一下-会很有趣。

敬请关注 ！！ (Stay Tuned !!)