Flink线上问题: The assigned slot container_xxx was removed
Flink线上问题: The assigned slot container_xxx was removed
客户现场使用Flink(on Yarn)进行数据抽取,Source是JDBC,Sink是Kafka,客户反映流程差不多跑10天左右就挂,让我看看.
环境:
Flink: 1.5.2
jdk: 1.8.0_25
Hadoop: 2.4.1
jobmanger和TaskManger都分配1G内存
首先我看了一下我们系统收集到的日志,有2段可能有用.
第一段:
2019-12-26 08:42:42,157 WARN akka.remote.ReliableDeliverySupervisor - Association with remote system [akka.tcp://flink@tdh04:46540] has failed, address is now gated for [50] ms. Reason: [Disassociated]
2019-12-26 08:42:42,877 INFO org.apache.flink.yarn.YarnResourceManager - Closing TaskExecutor connection container_1576488269936_0008_01_000008 because: Exception from container-launch.
Container id: container_e05_1576488269936_0008_01_000008
Exit code: 255
Stack trace: ExitCodeException exitCode=255: at org.apache.hadoop.util.Shell.runCommand(Shell.java:585)at org.apache.hadoop.util.Shell.run(Shell.java:482)at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:776)at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)at java.util.concurrent.FutureTask.run(FutureTask.java:262)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 255
第二段:
2019-12-26 08:42:42,877 INFO org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager - Unregister TaskManager f4660fcc70ee329e2427b5ed1245aa83 from the SlotManager.
2019-12-26 08:42:42,878 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: JDBC Source -> Timestamps/Watermarks -> sink_0_projection -> 数据输出_0_CheckOutputTypeFunction -> Sink: Unnamed (1/1) (96b3d4c2da227693ac34a8e8d2a4abea) switched from RUNNING to FAILED.
org.apache.flink.util.FlinkException: The assigned slot container_1576488269936_0008_01_000008_0 was removed.at org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlot(SlotManager.java:786)at org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlots(SlotManager.java:756)at org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.internalUnregisterTaskManager(SlotManager.java:948)at org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.unregisterTaskManager(SlotManager.java:372)at org.apache.flink.runtime.resourcemanager.ResourceManager.closeTaskManagerConnection(ResourceManager.java:793)at org.apache.flink.yarn.YarnResourceManager.lambda$onContainersCompleted$0(YarnResourceManager.java:339)at org.apache.flink.yarn.YarnResourceManager$$Lambda$212/930337248.run(Unknown Source)at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:332)at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:158)at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:70)at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.onReceive(AkkaRpcActor.java:142)at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.onReceive(FencedAkkaRpcActor.java:40)at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165)at akka.actor.Actor$class.aroundReceive(Actor.scala:502)at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)at akka.actor.ActorCell.invoke(ActorCell.scala:495)at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)at akka.dispatch.Mailbox.run(Mailbox.scala:224)at akka.dispatch.Mailbox.exec(Mailbox.scala:234)at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2019-12-26 08:42:42,879 INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Discarding checkpoint 82951 of job 42983ed24d98360747a8a535f2a3c8ba because: The assigned slot container_1576488269936_0008_01_000008_0 was removed.
2019-12-26 08:42:42,879 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job flow_1575462616880_0011 (42983ed24d98360747a8a535f2a3c8ba) switched from state RUNNING to FAILING.
org.apache.flink.util.FlinkException: The assigned slot container_1576488269936_0008_01_000008_0 was removed.at org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlot(SlotManager.java:786)at org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlots(SlotManager.java:756)at org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.internalUnregisterTaskManager(SlotManager.java:948)at org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.unregisterTaskManager(SlotManager.java:372)at org.apache.flink.runtime.resourcemanager.ResourceManager.closeTaskManagerConnection(ResourceManager.java:793)at org.apache.flink.yarn.YarnResourceManager.lambda$onContainersCompleted$0(YarnResourceManager.java:339)at org.apache.flink.yarn.YarnResourceManager$$Lambda$212/930337248.run(Unknown Source)at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:332)at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:158)at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:70)at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.onReceive(AkkaRpcActor.java:142)at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.onReceive(FencedAkkaRpcActor.java:40)at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165)at akka.actor.Actor$class.aroundReceive(Actor.scala:502)at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)at akka.actor.ActorCell.invoke(ActorCell.scala:495)at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)at akka.dispatch.Mailbox.run(Mailbox.scala:224)at akka.dispatch.Mailbox.exec(Mailbox.scala:234)at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2019-12-26 08:42:42,882 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Try to restart or fail the job flow_1575462616880_0011 (42983ed24d98360747a8a535f2a3c8ba) if no longer possible.
2019-12-26 08:42:42,882 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job flow_1575462616880_0011 (42983ed24d98360747a8a535f2a3c8ba) switched from state FAILING to FAILED.
org.apache.flink.util.FlinkException: The assigned slot container_1576488269936_0008_01_000008_0 was removed.at org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlot(SlotManager.java:786)at org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlots(SlotManager.java:756)at org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.internalUnregisterTaskManager(SlotManager.java:948)at org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.unregisterTaskManager(SlotManager.java:372)at org.apache.flink.runtime.resourcemanager.ResourceManager.closeTaskManagerConnection(ResourceManager.java:793)at org.apache.flink.yarn.YarnResourceManager.lambda$onContainersCompleted$0(YarnResourceManager.java:339)at org.apache.flink.yarn.YarnResourceManager$$Lambda$212/930337248.run(Unknown Source)at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:332)at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:158)at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:70)at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.onReceive(AkkaRpcActor.java:142)at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.onReceive(FencedAkkaRpcActor.java:40)at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165)at akka.actor.Actor$class.aroundReceive(Actor.scala:502)at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)at akka.actor.ActorCell.invoke(ActorCell.scala:495)at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)at akka.dispatch.Mailbox.run(Mailbox.scala:224)at akka.dispatch.Mailbox.exec(Mailbox.scala:234)at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2019-12-26 08:42:42,882 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Could not restart the job flow_1575462616880_0011 (42983ed24d98360747a8a535f2a3c8ba) because the restart strategy prevented it.
org.apache.flink.util.FlinkException: The assigned slot container_1576488269936_0008_01_000008_0 was removed.at org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlot(SlotManager.java:786)at org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.removeSlots(SlotManager.java:756)at org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.internalUnregisterTaskManager(SlotManager.java:948)at org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager.unregisterTaskManager(SlotManager.java:372)at org.apache.flink.runtime.resourcemanager.ResourceManager.closeTaskManagerConnection(ResourceManager.java:793)at org.apache.flink.yarn.YarnResourceManager.lambda$onContainersCompleted$0(YarnResourceManager.java:339)at org.apache.flink.yarn.YarnResourceManager$$Lambda$212/930337248.run(Unknown Source)at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:332)at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:158)at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:70)at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.onReceive(AkkaRpcActor.java:142)at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.onReceive(FencedAkkaRpcActor.java:40)at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165)at akka.actor.Actor$class.aroundReceive(Actor.scala:502)at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)at akka.actor.ActorCell.invoke(ActorCell.scala:495)at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)at akka.dispatch.Mailbox.run(Mailbox.scala:224)at akka.dispatch.Mailbox.exec(Mailbox.scala:234)at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
看完日志能得到的信息非常有限,就知道了container被移除了,至于为什么被移除还不知道.接下来去Yarn上看一下日志.(Yarn要开启日志聚合,第一次让我看问题的时候日志聚合没有开,我把日志聚合打开后告诉他有问题之后再找我)
Yarn上有用的日志:
jobmanager.out
2019-12-26 08:42:44,233 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - RECEIVED SIGNAL 15: SIGTERM. Shutting down as requested.
taskmanager.err
SEVERE: Failed to resolve default logging config file: config/java.util.logging.properties
Uncaught error from thread [flink-akka.actor.default-dispatcher-4]: GC overhead limit exceeded, shutting down JVM since 'akka.jvm-exit-on-fatal-error' is enabled for for ActorSystem[flink]
java.lang.OutOfMemoryError: GC overhead limit exceededat sun.reflect.AccessorGenerator.emitConstructor(AccessorGenerator.java:429)at sun.reflect.MethodAccessorGenerator.generate(MethodAccessorGenerator.java:379)at sun.reflect.MethodAccessorGenerator.generateSerializationConstructor(MethodAccessorGenerator.java:112)at sun.reflect.ReflectionFactory.newConstructorForSerialization(ReflectionFactory.java:340)at java.io.ObjectStreamClass.getSerializableConstructor(ObjectStreamClass.java:1376)at java.io.ObjectStreamClass.access$1500(ObjectStreamClass.java:72)at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:493)at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:468)at java.security.AccessController.doPrivileged(Native Method)at java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:468)at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:365)at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:602)at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1623)at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1518)at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1623)at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1518)at java.io.ObjectInputStream.readClass(ObjectInputStream.java:1484)at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1334)at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)at org.apache.flink.runtime.rpc.messages.RemoteRpcInvocation$MethodInvocation.readObject(RemoteRpcInvocation.java:204)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:483)at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1896)at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:502)at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:489)at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:477)
我按时间先后整理了一下:
2019-12-26 08:42:42,877 Container退出
2019-12-26 08:42:44.084 kill job
2019-12-26 08:42:44,233 ClusterEntrypoint - RECEIVED SIGNAL 15
2019-12-26 08:42:45,005 FINISH_APPLICATION sent to absent application application_1576488269936_0008
首先是TaskManger的Container退出,其实这时候任务就失败了,由于任务异常结束,系统主动kill Yarn上的任务,所以SIGNAL 15其实是我们自己发的,下面那条其是在Yarn上kill一个已经不存在的任务时发出的警告.
由于没有找到明显导致Container退出的原因,结合以下信息:
- 任务可以正常启动
- 几乎周期性的失败
- akka报的OutOfMemory
猜测可能是存在内存泄露,然后我就去业务代码(不是我写的)里看,看到JDBC PreparedStatement每次都弄个新的,并且没有close,我猜测可能就是它导致的,然后写了一段Demo,如下:
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.ResultSet;public class JDBC {public static void main(String[] args) throws Exception {Class.forName("com.mysql.jdbc.Driver");Connection conn = DriverManager.getConnection("jdbc:mysql://localhost:3306/test", "root", "123456");PreparedStatement preparedStatement;while (true) {preparedStatement = conn.prepareStatement("select count(*) from table_a");ResultSet resultSet = preparedStatement.executeQuery();while (resultSet.next()) {resultSet.getObject(1);}//preparedStatement.close();//Thread.sleep(10);}}
}
JVM参数:-Xmx100M
使用Visual VM观察内存变化,启动程序后很快就OOM了,所以加了个sleep,通过Visual VM观察老年代一直在增长,后来发生了GC但是也不管用还是OOM了.
更改代码把preparedStatement关闭之后老年代虽然也可以几乎涨满,但是GC过后内存就下来了.
如果不是任务启动的时候因为其他异常比如ClassNotFound导致的The assigned slot container_xxx was removed,并且周期性的出现问题,要优先考虑一下内存泄露.
欢迎关注公众号:大数据开发者
Flink线上问题: The assigned slot container_xxx was removed相关推荐
- 【Flink】FLink assigned slot xx was removed
1.概述 flink报错 org.apache.flink.util.FlinkException: The assigned slot container_e08_1539148828017_159 ...
- 线上flink任务重启报错(Hadoop问题)java.lang.NumberFormatException: For input string: “30s“
线上Flink任务重启时,提交任务失败,具体报错如下: 2021-12-31 18:09:19 java.lang.NumberFormatException: For input string: & ...
- Android App 线上热修复方案
热修复一词恐怕最早应用在微软.为了巩固其windows系统和office的市场占有率,微软开发并维护了一套线上修复方案,用于修复漏洞及特定问题(LDR),避免延续到发版解决(GDR),详见HotFix ...
- Android App 线上热修复方案Xposed
热修复一词恐怕最早应用在微软.为了巩固其windows系统和office的市场占有率,微软开发并维护了一套线上修复方案,用于修复漏洞及特定问题(LDR),避免延续到发版解决(GDR),详见HotFix ...
- 全球首次!阿里云把 Apache 顶级开源项目会议搬到线上
2019 年 12 月,Flink Forward 亚洲站邀请了来自阿里巴巴.字节跳动.滴滴.美团点评.小米.bilibili.快手等一线厂商 45 位技术大佬分享其技术心得和实战干货,以及 Flin ...
- 内存地址 哪个程序_记一次排查线上程序内存的忽高忽低,又是大集合惹祸了...
一:背景 1. 讲故事 昨天继续还技术债,优化一轮后的程序拉到线上后内存继续忽高忽低,低的时候20G,高的时候30G,过了一会又下降了几个G,毫无疑问,程序中有什么集合或者什么操作占用了大量内存,所以 ...
- 今日直播 | Apache Hudi x Apache Pulsar Meetup线上专场如期而至 大咖齐聚
简介:Apache Hudi 与 Apache Pulsar 联合 Meetup 线上专场将于2021 年 8 月 30 日(今天) 14:00开启直播,你准备好了吗? Apache Hudi 与 A ...
- 记一次排查线上程序内存的忽高忽低,又是大集合惹祸了
一:背景 1. 讲故事 昨天继续还技术债,优化一轮后的程序拉到线上后内存继续忽高忽低,低的时候20G,高的时候30G,过了一会又下降了几个G,毫无疑问,程序中有什么集合或者什么操作占用了大量内存,所以 ...
- streamx如何支持我司线上实时作业
导读:本文主要带来 streamx 在我司中实时任务的应用,我们的部署方式是onyarn的模式,然后使用的版本是自己编译的1.2.1,都是基于Flink做的任务开发. 本文通过一下几个点展开: joy ...
最新文章
- vi 技巧和诀窍:令人刮目相看的 10 个超酷命令
- C/C++中程序内存区域划分大总结
- 我与微软的不解之缘 - 我的Insider Dev Tour 2019讲师之旅
- git clone 时候出现Please make sure you have the correct access rights and the repository exists.
- vmware-vmx.exe无法结束进程_孤儿进程与僵尸进程产生原理分析,以及终极解决方案案例实现...
- 可蠕虫 DarkRadiation 勒索软件瞄准 Linux 和 Docker 实例
- 实现一个输入程序,接收从 键盘读入的字符串。当字符串中所含字符个数少于程序设定的上限时,输出这个字符串;否则抛出MyStringException1异常,在异常处理中要求重新输入新的字符串或者中断程序
- 机器学习面试-处理回归问题
- ORACLE在线切换undo表空间
- 贝叶斯网络分析kaggle泰坦尼克号数据
- BigGAN高保真自然图像合成的大规模GAN训练
- waves效果器_学会EQ效果器,浑厚亮嗓又大气
- 跨专业考计算机研究生有专业限制吗,我想跨专业考计算机专业研究生
- css3 translate属性
- 计算机的内存不足建议使用,别再说内存不足了!教你一招彻底解决,电脑运行加速3倍!...
- java安装及设置eclipse
- JavaScript面试题大全之基础面试题(附答案)
- MOGRT动态图标模板 爱情元素手绘爱心特效pr视频模板
- python 基于xgboost预测波士顿房价
- 走进就职演说幕后:大有深意的用词
热门文章
- 采集百度图片不显示 怎么办
- vscode中配置settting.json
- VS2010读取大恒相机图像并显示
- php实训ppt,php实训ppt
- PythonOCC基础使用:建模——倒角倒圆(二维+三维)
- 深度学习模型部署浅析
- 学习QT前需了解的知识
- Freeline的快速集成
- (翻译)Few-Shot Object Detection with Attention-RPN and Multi-Relation Detector具有注意力RPN和多关系检测器的小样本目标检测
- ADIS16400/ADIS16405带磁力计的三轴惯性传感器(3)