Android WebRTC实现音视频对讲
2024-04-06 17:54:46
WebRTC
- 新建Android工程 并添加依赖
- 渲染视频的view
- 初始化控件
- 初始化PeerConnectionFactor
- 创建PeerConnection
- 注册事件监听
- 发送offer
- 收到Offer
- 发送answer
- 收到answer
- Candidate变化
- 服务端代码
- 找到自己电脑的ip
- Android端P2P完成通信,发现延迟比较高
慕课网课程webrtc入门学习后的总结
- github代码
新建Android工程 并添加依赖
- build.gradle
android {... compileOptions {sourceCompatibility = '1.8'targetCompatibility = '1.8'}
}
dependencies {implementation 'io.socket:socket.io-client:1.0.0'//webrtc库 ,可自己编译implementation 'org.webrtc:google-webrtc:1.0.+'//Android 6.0+ 动态权限申请库implementation 'pub.devrel:easypermissions:1.1.3'
}
- AndroidManifest.xml 记得申请权限
<uses-feature android:name="android.hardware.camera" /><uses-feature android:name="android.hardware.camera.autofocus" /><uses-featureandroid:glEsVersion="0x00020000"android:required="true" /><uses-permission android:name="android.permission.CAMERA" /><uses-permission android:name="android.permission.CHANGE_NETWORK_STATE" /><uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" /><uses-permission android:name="android.permission.RECORD_AUDIO" /><uses-permission android:name="android.permission.BLUETOOTH" /><uses-permission android:name="android.permission.INTERNET" /><uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" /><uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" /><uses-permission android:name="android.permission.READ_PHONE_STATE" /><uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
- Activity里面动态申请权限
String[] perms = {Manifest.permission.CAMERA, Manifest.permission.RECORD_AUDIO};if (!EasyPermissions.hasPermissions(this, perms)) {EasyPermissions.requestPermissions(this, "Need permissions for camera & microphone", 0, perms);}@Overridepublic void onRequestPermissionsResult(int requestCode, String[] permissions, int[] grantResults) {super.onRequestPermissionsResult(requestCode, permissions, grantResults);EasyPermissions.onRequestPermissionsResult(requestCode, permissions, grantResults, this);}
渲染视频的view
<?xml version="1.0" encoding="utf-8"?>
<FrameLayout xmlns:android="http://schemas.android.com/apk/res/android"android:layout_width="match_parent"android:layout_height="match_parent" >// 渲染本地视频<org.webrtc.SurfaceViewRendererandroid:id="@+id/LocalSurfaceView"android:layout_width="wrap_content"android:layout_height="wrap_content"android:layout_gravity="center" />
//渲染远程视频<org.webrtc.SurfaceViewRendererandroid:id="@+id/RemoteSurfaceView"android:layout_width="120dp"android:layout_height="160dp"android:layout_gravity="top|end"android:layout_margin="16dp"/>
//日志打印<TextViewandroid:id="@+id/LogcatView"android:layout_width="wrap_content"android:layout_height="wrap_content"android:layout_margin="5dp"android:textColor="@android:color/white"android:layout_gravity="top|start" /></FrameLayout>
初始化控件
private void initView() {mLogcatView = findViewById(R.id.LogcatView);mLocalSurfaceView = findViewById(R.id.LocalSurfaceView);mRemoteSurfaceView = findViewById(R.id.RemoteSurfaceView);mRootEglBase = EglBase.create();//本地视频流渲染初始化mLocalSurfaceView.init(mRootEglBase.getEglBaseContext(), null);mLocalSurfaceView.setScalingType(RendererCommon.ScalingType.SCALE_ASPECT_FILL);mLocalSurfaceView.setMirror(true);mLocalSurfaceView.setEnableHardwareScaler(false);//远程视频流渲染初始化mRemoteSurfaceView.init(mRootEglBase.getEglBaseContext(), null);mRemoteSurfaceView.setScalingType(RendererCommon.ScalingType.SCALE_ASPECT_FILL);mRemoteSurfaceView.setMirror(true);mRemoteSurfaceView.setEnableHardwareScaler(true);mRemoteSurfaceView.setZOrderMediaOverlay(true);}
初始化PeerConnectionFactor
/*** 创建PC工厂** @return*/private PeerConnectionFactory createPeerConnectionFactory() {//创建视频编解码工厂VideoEncoderFactory encoderFactory = new DefaultVideoEncoderFactory(mRootEglBase.getEglBaseContext(),false,true);VideoDecoderFactory decoderFactory = new DefaultVideoDecoderFactory(mRootEglBase.getEglBaseContext());//初始化PC工厂参数PeerConnectionFactory.InitializationOptions initializationOptions = PeerConnectionFactory.InitializationOptions.builder(this).setEnableInternalTracer(true).createInitializationOptions();PeerConnectionFactory.initialize(initializationOptions);PeerConnectionFactory.Builder builder = PeerConnectionFactory.builder().setVideoDecoderFactory(decoderFactory).setVideoEncoderFactory(encoderFactory);builder.setOptions(null);return builder.createPeerConnectionFactory();}private void initPeerConnectionFactor() {mPeerConnectionFactory = createPeerConnectionFactory();// NOTE: this _must_ happen while PeerConnectionFactory is alive!Logging.enableLogToDebugOutput(Logging.Severity.LS_VERBOSE);mSurfaceTextureHelper = SurfaceTextureHelper.create("CaptureThread", mRootEglBase.getEglBaseContext());//创建视频源 ,参数是否截屏VideoSource videoSource = mPeerConnectionFactory.createVideoSource(false);//创建捕获视频流mVideoCapturer = createVideoCapturer();//初始化视频铺货器mVideoCapturer.initialize(mSurfaceTextureHelper, getApplicationContext(), videoSource.getCapturerObserver());//视频源和视频轨绑定mVideoTrack = mPeerConnectionFactory.createVideoTrack(VIDEO_TRACK_ID, videoSource);mVideoTrack.setEnabled(true);//视频流用本地View控件显示mVideoTrack.addSink(mLocalSurfaceView);//创建音频源AudioSource audioSource = mPeerConnectionFactory.createAudioSource(new MediaConstraints());mAudioTrack = mPeerConnectionFactory.createAudioTrack(AUDIO_TRACK_ID, audioSource);mAudioTrack.setEnabled(true);}
创建PeerConnection
public PeerConnection createPeerConnection() {Log.i(TAG, "Create PeerConnection ...");LinkedList<PeerConnection.IceServer> iceServers = new LinkedList<>();//搭建好stun服务器可以加上配置/*PeerConnection.IceServer iceServer = PeerConnection.IceServer.builder("").setPassword("").setUsername("").createIceServer();iceServers.add(iceServer);*/PeerConnection.RTCConfiguration rtcConfig = new PeerConnection.RTCConfiguration(iceServers);// TCP candidates are only useful when connecting to a server that supports// ICE-TCP.rtcConfig.tcpCandidatePolicy = PeerConnection.TcpCandidatePolicy.DISABLED;//rtcConfig.bundlePolicy = PeerConnection.BundlePolicy.MAXBUNDLE;//rtcConfig.rtcpMuxPolicy = PeerConnection.RtcpMuxPolicy.REQUIRE;rtcConfig.continualGatheringPolicy = PeerConnection.ContinualGatheringPolicy.GATHER_CONTINUALLY;// Use ECDSA encryption.//rtcConfig.keyType = PeerConnection.KeyType.ECDSA;// Enable DTLS for normal calls and disable for loopback calls.rtcConfig.enableDtlsSrtp = true;//rtcConfig.sdpSemantics = PeerConnection.SdpSemantics.UNIFIED_PLAN;PeerConnection connection = mPeerConnectionFactory.createPeerConnection(rtcConfig, mPeerConnectionObserver);if (connection == null) {return null;}//pc音视频轨List<String> mediaStreamLabels = Collections.singletonList("ARDAMS");connection.addTrack(mVideoTrack, mediaStreamLabels);connection.addTrack(mAudioTrack, mediaStreamLabels);return connection;}
注册事件监听
这里的事件是client和server使用socket.io触发的一些事件
- client
- joined --发起join后Server会回joined
- leaved–发送leave后Server会回leaved
- otherjoin – 当另一个client加入房间后 ,server会回otherjoin
- bye–当另一个client发送leave,server会回bye告诉我
- full – 当房间满了 ,这时候加入server会回full
- message – sdp交换的消息监听 ,offer answer candidate等事件
- server
- connection
- join – 收到client 加入房间的事件
- leave – 收到client离开房间的事件
- connection
public interface OnSignalEventListener {void onConnected();void onConnecting();void onDisconnected();void onUserJoined(String roomName, String userID);void onUserLeaved(String roomName, String userID);void onRemoteUserJoined(String roomName);void onRemoteUserLeaved(String roomName, String userID);void onRoomFull(String roomName, String userID);void onMessage(JSONObject message);}
- 服务端server
io.sockets.on('connection', (socket)=> {socket.on('message', (room, data)=>{socket.to(room).emit('message',room, data);});socket.on('join', (room)=>{socket.join(room);var myRoom = io.sockets.adapter.rooms[room]; var users = (myRoom)? Object.keys(myRoom.sockets).length : 0;logger.debug('the user number of room is: ' + users);if(users < USERCOUNT){socket.emit('joined', room, socket.id); //发给除自己之外的房间内的所有人if(users > 1){socket.to(room).emit('otherjoin', room, socket.id);}}else{socket.leave(room); socket.emit('full', room, socket.id);}//socket.emit('joined', room, socket.id); //发给自己//socket.broadcast.emit('joined', room, socket.id); //发给除自己之外的这个节点上的所有人//io.in(room).emit('joined', room, socket.id); //发给房间内的所有人});socket.on('leave', (room)=>{var myRoom = io.sockets.adapter.rooms[room]; var users = (myRoom)? Object.keys(myRoom.sockets).length : 0;logger.debug('the user number of room is: ' + (users-1));//socket.emit('leaved', room, socket.id);//socket.broadcast.emit('leaved', room, socket.id);socket.to(room).emit('bye', room, socket.id);socket.emit('leaved', room, socket.id);//io.in(room).emit('leaved', room, socket.id);});});
- 客户端client
//发送消息
public void joinRoom(String url, String roomName) {Log.i(TAG, "joinRoom: " + url + ", " + roomName);try {mSocket = IO.socket(url);mSocket.connect();} catch (URISyntaxException e) {e.printStackTrace();return;}//mUserId = userId;mRoomName = roomName;listenSignalEvents();mSocket.emit("join", mRoomName);}public void leaveRoom() {Log.i(TAG, "leaveRoom: " + mRoomName);if (mSocket == null) {return;}mSocket.emit("leave", mRoomName);mSocket.close();mSocket = null;}public void sendMessage(JSONObject message) {Log.i(TAG, "broadcast: " + message);if (mSocket == null) {return;}mSocket.emit("message", mRoomName, message);}
//接收消息,监听服务器的消息
mSocket.on(Socket.EVENT_CONNECT_ERROR, new Emitter.Listener() {@Overridepublic void call(Object... args) {Log.e(TAG, "onConnectError: " + args);}});mSocket.on(Socket.EVENT_ERROR, new Emitter.Listener() {@Overridepublic void call(Object... args) {Log.e(TAG, "onError: " + args);}});mSocket.on(Socket.EVENT_CONNECT, new Emitter.Listener() {@Overridepublic void call(Object... args) {String sessionId = mSocket.id();Log.i(TAG, "onConnected");if (mOnSignalEventListener != null) {mOnSignalEventListener.onConnected();}}});mSocket.on(Socket.EVENT_CONNECTING, new Emitter.Listener() {@Overridepublic void call(Object... args) {Log.i(TAG, "onConnecting");if (mOnSignalEventListener != null) {mOnSignalEventListener.onConnecting();}}});mSocket.on(Socket.EVENT_DISCONNECT, new Emitter.Listener() {@Overridepublic void call(Object... args) {Log.i(TAG, "onDisconnected");if (mOnSignalEventListener != null) {mOnSignalEventListener.onDisconnected();}}});mSocket.on("joined", new Emitter.Listener() {@Overridepublic void call(Object... args) {String roomName = (String) args[0];String userId = (String) args[1];if (/*!mUserId.equals(userId) &&*/ mOnSignalEventListener != null) {//mOnSignalEventListener.onRemoteUserJoined(userId);mOnSignalEventListener.onUserJoined(roomName, userId);}//Log.i(TAG, "onRemoteUserJoined: " + userId);Log.i(TAG, "onUserJoined, room:" + roomName + "uid:" + userId);}});mSocket.on("leaved", new Emitter.Listener() {@Overridepublic void call(Object... args) {String roomName = (String) args[0];String userId = (String) args[1];if (/*!mUserId.equals(userId) &&*/ mOnSignalEventListener != null) {//mOnSignalEventListener.onRemoteUserLeft(userId);mOnSignalEventListener.onUserLeaved(roomName, userId);}Log.i(TAG, "onUserLeaved, room:" + roomName + "uid:" + userId);}});mSocket.on("otherjoin", new Emitter.Listener() {@Overridepublic void call(Object... args) {String roomName = (String) args[0];String userId = (String) args[1];if (mOnSignalEventListener != null) {mOnSignalEventListener.onRemoteUserJoined(roomName);}Log.i(TAG, "onRemoteUserJoined, room:" + roomName + "uid:" + userId);}});mSocket.on("bye", new Emitter.Listener() {@Overridepublic void call(Object... args) {String roomName = (String) args[0];String userId = (String) args[1];if (mOnSignalEventListener != null) {mOnSignalEventListener.onRemoteUserLeaved(roomName, userId);}Log.i(TAG, "onRemoteUserLeaved, room:" + roomName + "uid:" + userId);}});mSocket.on("full", new Emitter.Listener() {@Overridepublic void call(Object... args) {//释放资源mSocket.disconnect();mSocket.close();mSocket = null;String roomName = (String) args[0];String userId = (String) args[1];if (mOnSignalEventListener != null) {mOnSignalEventListener.onRoomFull(roomName, userId);}Log.i(TAG, "onRoomFull, room:" + roomName + "uid:" + userId);}});mSocket.on("message", new Emitter.Listener() {@Overridepublic void call(Object... args) {String roomName = (String)args[0];JSONObject msg = (JSONObject) args[1];if (mOnSignalEventListener != null) {mOnSignalEventListener.onMessage(msg);}Log.i(TAG, "onMessage, room:" + roomName + "data:" + msg);}});
发送offer
MediaConstraints mediaConstraints = new MediaConstraints();mediaConstraints.mandatory.add(new MediaConstraints.KeyValuePair("OfferToReceiveAudio", "true"));mediaConstraints.mandatory.add(new MediaConstraints.KeyValuePair("OfferToReceiveVideo", "true"));//不打开dtls无法和web端通信mediaConstraints.optional.add(new MediaConstraints.KeyValuePair("DtlsSrtpKeyAgreement", "true"));mPeerConnection.createOffer(new SimpleSdpObserver() {@Overridepublic void onCreateSuccess(SessionDescription sessionDescription) {Log.i(TAG, "Create local offer success: \n" + sessionDescription.description);mPeerConnection.setLocalDescription(new SimpleSdpObserver(), sessionDescription);JSONObject message = new JSONObject();try {message.put("type", "offer");message.put("sdp", sessionDescription.description);SignalClient.getInstance().sendMessage(message);} catch (JSONException e) {e.printStackTrace();}}}, mediaConstraints);
收到Offer
private void onRemoteOfferReceived(JSONObject message) {logcatOnUI("Receive Remote Call ...");if (mPeerConnection == null) {mPeerConnection = createPeerConnection();}try {String description = message.getString("sdp");mPeerConnection.setRemoteDescription(new SimpleSdpObserver(),new SessionDescription(SessionDescription.Type.OFFER,description));doAnswerCall();//发送answer} catch (JSONException e) {e.printStackTrace();}}
发送answer
MediaConstraints sdpMediaConstraints = new MediaConstraints();Log.i(TAG, "Create answer ...");mPeerConnection.createAnswer(new SimpleSdpObserver() {@Overridepublic void onCreateSuccess(SessionDescription sessionDescription) {Log.i(TAG, "Create answer success !");mPeerConnection.setLocalDescription(new SimpleSdpObserver(),sessionDescription);JSONObject message = new JSONObject();try {message.put("type", "answer");message.put("sdp", sessionDescription.description);SignalClient.getInstance().sendMessage(message);} catch (JSONException e) {e.printStackTrace();}}}, sdpMediaConstraints);
收到answer
private void onRemoteAnswerReceived(JSONObject message) {try {String description = message.getString("sdp");mPeerConnection.setRemoteDescription(new SimpleSdpObserver(),new SessionDescription(SessionDescription.Type.ANSWER,description));} catch (JSONException e) {e.printStackTrace();}}
offer 和answer后就完成了信令交互
Candidate变化
//PeerConnection.Observer 监听到Candidate变化了 ,发送Candidate
@Overridepublic void onIceCandidate(IceCandidate iceCandidate) {Log.i(TAG, "onIceCandidate: " + iceCandidate);try {JSONObject message = new JSONObject();//message.put("userId", RTCSignalClient.getInstance().getUserId());message.put("type", "candidate");message.put("label", iceCandidate.sdpMLineIndex);message.put("id", iceCandidate.sdpMid);message.put("candidate", iceCandidate.sdp);SignalClient.getInstance().sendMessage(message);} catch (JSONException e) {e.printStackTrace();}}
//收到Candidate
private void onRemoteCandidateReceived(JSONObject message) {try {IceCandidate remoteIceCandidate =new IceCandidate(message.getString("id"),message.getInt("label"),message.getString("candidate"));mPeerConnection.addIceCandidate(remoteIceCandidate);} catch (JSONException e) {e.printStackTrace();}}
服务端代码
- server.js
//我注释掉了https服务 ,使用http
'use strict'var log4js = require('log4js');
var http = require('http');
//var https = require('https');
var fs = require('fs');
var socketIo = require('socket.io');var express = require('express');
var serveIndex = require('serve-index');var USERCOUNT = 3;log4js.configure({appenders: {file: {type: 'file',filename: 'app.log',layout: {type: 'pattern',pattern: '%r %p - %m',}}},categories: {default: {appenders: ['file'],level: 'debug'}}
});var logger = log4js.getLogger();var app = express();
//app.use(serveIndex('./public')); //这些是放web端代码的地方 ,我只测试Android端
//app.use(express.static('./public'));//http server
var http_server = http.createServer(app);
http_server.listen(80, '0.0.0.0'); //监听80端口
//https_server.listen(443, '0.0.0.0');
//var options = {
// key : fs.readFileSync('./cert/2280243_starcloud.club.key'),
// cert: fs.readFileSync('./cert/2280243_starcloud.club.pem')
//}//https server
//var https_server = https.createServer(options, app);
var io = socketIo.listen(http_server);io.sockets.on('connection', (socket)=> {socket.on('message', (room, data)=>{socket.to(room).emit('message',room, data);});socket.on('join', (room)=>{socket.join(room);var myRoom = io.sockets.adapter.rooms[room]; var users = (myRoom)? Object.keys(myRoom.sockets).length : 0;logger.debug('the user number of room is: ' + users);if(users < USERCOUNT){socket.emit('joined', room, socket.id); //发给除自己之外的房间内的所有人if(users > 1){socket.to(room).emit('otherjoin', room, socket.id);}}else{socket.leave(room); socket.emit('full', room, socket.id);}//socket.emit('joined', room, socket.id); //发给自己//socket.broadcast.emit('joined', room, socket.id); //发给除自己之外的这个节点上的所有人//io.in(room).emit('joined', room, socket.id); //发给房间内的所有人});socket.on('leave', (room)=>{var myRoom = io.sockets.adapter.rooms[room]; var users = (myRoom)? Object.keys(myRoom.sockets).length : 0;logger.debug('the user number of room is: ' + (users-1));//socket.emit('leaved', room, socket.id);//socket.broadcast.emit('leaved', room, socket.id);socket.to(room).emit('bye', room, socket.id);socket.emit('leaved', room, socket.id);//io.in(room).emit('leaved', room, socket.id);});
});
- 启动服务 , 这里需要安装node.js
找到自己电脑的ip
- 进入房间
Android端P2P完成通信,发现延迟比较高
Android WebRTC实现音视频对讲相关推荐
- Android 平台点对点音视频对讲
Communication Android平台 点对点 音视频对讲 项目链接 https://github.com/yuzhihui170/Communication 本项目提供Android平台点对 ...
- Android端WebRTC本地音视频采集流程源码分析
WebRTC源码版本为:org.webrtc:google-webrtc:1.0.32006 本文仅分析Java层源码,在分析之前,先说明一下一些重要类的基本概念. MediaSource:WebRT ...
- Android端实时音视频开发指南
简介 yun2win-sdk-Android提供Android端实时音视频完整解决方案,方便客户快速集成实时音视频功能. SDK 提供的能力如下: 发起 加入 AVClient Channel AVM ...
- 【复】基于 WebRTC 的音视频在线监考模块的设计与实现（上）
文章目录 前言 什么是 WebRTC? WebRTC 架构 WebRTC 通讯内容 WebRTC 通讯协议 WebRTC 连接建立过程 后记 前言 最近在做关于考试系统的项目,其中有一项需求分析是要做 ...
- Android移动端音视频的快速开发教程（五）
接 Android移动端音视频的快速开发教程(四) 3.3. 数据传输事件接口 3.3.1. 接口定义 package com.bairuitech.anychat; // 数据传输通知接口 pub ...
- 基于WebRTC实现音视频及数据通信
文章目录 前言 一.WebRTC的组成? 二.信令交换的方式 三.会话描述 四.客户端应用 1.HTML 2.JavaScript 五.效果演示 六.项目地址 总结 前言 刚写了篇基于WebRTC使用 ...
- 【原理+实战+视频+源码】抖音，快手大热背后——Android 贴心的音视频学习指南来咯！
导语 Android 音视频开发这块目前的确没有比较系统的教程或者书籍,网上的博客文章也都是比较零散的.只能通过一点点的学习和积累把这块的知识串联积累起来. 音视频的开发,往往是比较难的,而这个比较难 ...
- WebRTC实时音视频技术基础：基本架构和协议栈
概述 本文主要介绍WebRTC的架构和协议栈. 最基本的三角形WebRTC架构 为了便于理解,我们来看一个最基本的三角形WebRTC架构(见下图): 在这个架构中,移动电话用"浏览器M&qu ...
- 【WebRTC---入门篇】（十八）WebRTC非音视频数据传输
WebRTC传输非音视频重要API createDataChannel options ordered 在传输非音视频的时候是否是按序到达的. maxPacketLifeTime/maxRetrans ...
最新文章
- mysql外键写了会怎么样_mysql使用外键会影响性能吗
- DFS:深入优先搜索 POJ-2386 Lake Counting
- dropdownlist绑定的二种方法
- 我对香港数字生活的一些观察
- android 判断 飞行模式,如何在Android上检测飞行模式？
- [css] 使用css如何设置背景虚化？
- 第六十九期： 漫画说算法之什么是一致性哈希？
- 【java】多线程博客积累
- 如何修复“ DNS_PROBE_FINISHED_NXDOMAIN”错误
- 计算机考研408每日一题 day165
- MariaDB安装报1067错误解决方式
- 综合项目之闪讯破解（三）之 如何用C++实现PPPOE拨号
- 用照片进行三维模型重建
- 土方工程量计算表格excel_土方方格网计算表格excel.xls
- 哈希碰撞，改变世界的原力
- 如何拉取钉钉的外出、出差审批单
- error: C99 designator ‘personName’ outside aggregate initializer
- Silvaco TCAD仿真8——网格mesh的意义（举例说明）
- 背包模块的设计(日常任务模块, 武器排行榜, 战术, 英雄战斗力, 活动模块)
- ViewDragHelper实战，实现滑动解锁