在ARM板子上把玩Tensorflow Lite
前几天Google的IO大会上发布的ML Kit,ML Kit为端上部署深度学习模型提供了一套完整的解决方案,本地运行、云端都支持。里面本地部署用到的就是Tensorflow lite。
Tensorflow Lite是在Google去年IO大会上发表的,目前Tensorflow Lite也还在不断的完善迭代中。
Tensorflow Lite在Android和iOS上部署官网有比较详细的介绍以及对应的Demo。而对于ARM板子上的部署及测试,官网及网上的资料则相对较少。本文主要描述如何把Tensorflow Lite编译到ARM板子上,并运行相应的Demo。
0.准备工作:在Ubuntu上准备ARM的交叉编译环境
可以通过 apt-get install
直接在ubuntu上安装交叉编译环境
sudo apt-get install g++-arm-linux-gnueabihf
sudo apt-get install -y gcc-arm-linux-gnueabihf
1.下载Tensorflow源码
git clone https://github.com/tensorflow/tensorflow
2.下载Tensorflow相关依赖包
先cd到Tensorflow工程的根目录,然后执行下面的脚本
./tensorflow/contrib/lite/download_dependencies.sh
3.编译Tensorflow Lite
注意./tensorflow/contrib/lite/build_rpi_lib.sh
中的目标编译平台是ARMV7,这里最好不要改,即使你的目标平台是ARMv8。改为ARMv8能也能编译通过,但是貌似没有把优化编译进去,运行起来会非常慢,亲测。
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================set -eSCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$SCRIPT_DIR/../../.."#change CC_PREFIX if u need
CC_PREFIX=arm-linux-gnueabihf- make -j 3 -f tensorflow/contrib/lite/Makefile TARGET=RPI TARGET_ARCH=armv7
在根目录下运行该脚本,如下:
./tensorflow/contrib/lite/build_rpi_lib.sh
编译结束,会在tensorflow/contrib/lite/gen/lib/rpi_armv7
目录下产生libtensorflow-lite.a
静态库。
4.编译 label_image
Demo
第三步的 build_rpi_lib.sh
脚本实际是调用的 ./tensorflow/contrib/lite/Makefile
对Tensorflow Lite
源码进行编译,但是该 Makefile
并不能编译 tensorflow/contrib/lite/examples/label_image
目录下的Demo,所以需要修改 Makefile
把 label_image
的源码配置到Makefile中,修改方式可以参考 Makefile
里对 MINIMAL Demo
的配置。如果你不想自己改,下面是已经修改好的。
# Find where we're running from, so we can store generated files here.
ifeq ($(origin MAKEFILE_DIR), undefined)MAKEFILE_DIR := $(shell dirname $(realpath $(lastword $(MAKEFILE_LIST))))
endif# Try to figure out the host system
HOST_OS :=
ifeq ($(OS),Windows_NT)HOST_OS = WINDOWS
elseUNAME_S := $(shell uname -s)ifeq ($(UNAME_S),Linux)HOST_OS := LINUXendififeq ($(UNAME_S),Darwin)HOST_OS := OSXendif
endifARCH := $(shell if [[ $(shell uname -m) =~ i[345678]86 ]]; then echo x86_32; else echo $(shell uname -m); fi)# Where compiled objects are stored.
OBJDIR := $(MAKEFILE_DIR)/gen/obj/
BINDIR := $(MAKEFILE_DIR)/gen/bin/
LIBDIR := $(MAKEFILE_DIR)/gen/lib/
GENDIR := $(MAKEFILE_DIR)/gen/obj/# Settings for the host compiler.
CXX := $(CC_PREFIX)gcc
CXXFLAGS := --std=c++11 -O3 -DNDEBUG
CC := $(CC_PREFIX)gcc
CFLAGS := -O3 -DNDEBUG
LDOPTS :=
LDOPTS += -L/usr/local/lib
ARFLAGS := -rINCLUDES := \
-I. \
-I$(MAKEFILE_DIR)/../../../ \
-I$(MAKEFILE_DIR)/downloads/ \
-I$(MAKEFILE_DIR)/downloads/eigen \
-I$(MAKEFILE_DIR)/downloads/gemmlowp \
-I$(MAKEFILE_DIR)/downloads/neon_2_sse \
-I$(MAKEFILE_DIR)/downloads/farmhash/src \
-I$(MAKEFILE_DIR)/downloads/flatbuffers/include \
-I$(GENDIR)
# This is at the end so any globally-installed frameworks like protobuf don't
# override local versions in the source tree.
INCLUDES += -I/usr/local/includeLIBS := \
-lstdc++ \
-lpthread \
-lm \
-lz# If we're on Linux, also link in the dl library.
ifeq ($(HOST_OS),LINUX)LIBS += -ldl
endifinclude $(MAKEFILE_DIR)/ios_makefile.inc
include $(MAKEFILE_DIR)/rpi_makefile.inc# This library is the main target for this makefile. It will contain a minimal
# runtime that can be linked in to other programs.
LIB_NAME := libtensorflow-lite.a
LIB_PATH := $(LIBDIR)$(LIB_NAME)# A small example program that shows how to link against the library.
MINIMAL_PATH := $(BINDIR)minimal
LABEL_IMAGE_PATH :=$(BINDIR)label_imageMINIMAL_SRCS := \
tensorflow/contrib/lite/examples/minimal/minimal.cc
MINIMAL_OBJS := $(addprefix $(OBJDIR), \
$(patsubst %.cc,%.o,$(patsubst %.c,%.o,$(MINIMAL_SRCS))))LABEL_IMAGE_SRCS := \
tensorflow/contrib/lite/examples/label_image/label_image.cc \
tensorflow/contrib/lite/examples/label_image/bitmap_helpers.cc
LABEL_IMAGE_OBJS := $(addprefix $(OBJDIR), \
$(patsubst %.cc,%.o,$(patsubst %.c,%.o,$(LABEL_IMAGE_SRCS))))# What sources we want to compile, must be kept in sync with the main Bazel
# build files.CORE_CC_ALL_SRCS := \
$(wildcard tensorflow/contrib/lite/*.cc) \
$(wildcard tensorflow/contrib/lite/kernels/*.cc) \
$(wildcard tensorflow/contrib/lite/kernels/internal/*.cc) \
$(wildcard tensorflow/contrib/lite/kernels/internal/optimized/*.cc) \
$(wildcard tensorflow/contrib/lite/kernels/internal/reference/*.cc) \
$(wildcard tensorflow/contrib/lite/*.c) \
$(wildcard tensorflow/contrib/lite/kernels/*.c) \
$(wildcard tensorflow/contrib/lite/kernels/internal/*.c) \
$(wildcard tensorflow/contrib/lite/kernels/internal/optimized/*.c) \
$(wildcard tensorflow/contrib/lite/kernels/internal/reference/*.c) \
$(wildcard tensorflow/contrib/lite/downloads/farmhash/src/farmhash.cc) \
$(wildcard tensorflow/contrib/lite/downloads/fft2d/fftsg.c)
# Remove any duplicates.
CORE_CC_ALL_SRCS := $(sort $(CORE_CC_ALL_SRCS))
CORE_CC_EXCLUDE_SRCS := \
$(wildcard tensorflow/contrib/lite/*test.cc) \
$(wildcard tensorflow/contrib/lite/*/*test.cc) \
$(wildcard tensorflow/contrib/lite/*/*/*test.cc) \
$(wildcard tensorflow/contrib/lite/*/*/*/*test.cc) \
$(wildcard tensorflow/contrib/lite/kernels/test_util.cc) \
$(MINIMAL_SRCS) \
$(LABEL_IMAGE_SRCS)
# Filter out all the excluded files.
TF_LITE_CC_SRCS := $(filter-out $(CORE_CC_EXCLUDE_SRCS), $(CORE_CC_ALL_SRCS))
# File names of the intermediate files target compilation generates.
TF_LITE_CC_OBJS := $(addprefix $(OBJDIR), \
$(patsubst %.cc,%.o,$(patsubst %.c,%.o,$(TF_LITE_CC_SRCS))))
LIB_OBJS := $(TF_LITE_CC_OBJS)# For normal manually-created TensorFlow C++ source files.
$(OBJDIR)%.o: %.cc@mkdir -p $(dir $@)$(CXX) $(CXXFLAGS) $(INCLUDES) -c $< -o $@# For normal manually-created TensorFlow C++ source files.
$(OBJDIR)%.o: %.c@mkdir -p $(dir $@)$(CC) $(CCFLAGS) $(INCLUDES) -c $< -o $@# The target that's compiled if there's no command-line arguments.
all: $(LIB_PATH) $(MINIMAL_PATH) $(LABEL_IMAGE_PATH)# Gathers together all the objects we've compiled into a single '.a' archive.
$(LIB_PATH): $(LIB_OBJS)@mkdir -p $(dir $@)$(AR) $(ARFLAGS) $(LIB_PATH) $(LIB_OBJS)$(MINIMAL_PATH): $(MINIMAL_OBJS) $(LIB_PATH)@mkdir -p $(dir $@)$(CXX) $(CXXFLAGS) $(INCLUDES) \-o $(MINIMAL_PATH) $(MINIMAL_OBJS) \$(LIBFLAGS) $(LIB_PATH) $(LDFLAGS) $(LIBS)$(LABEL_IMAGE_PATH): $(LABEL_IMAGE_OBJS) $(LIB_PATH)@mkdir -p $(dir $@)$(CXX) $(CXXFLAGS) $(INCLUDES) \-o $(LABEL_IMAGE_PATH) $(LABEL_IMAGE_OBJS) \$(LIBFLAGS) $(LIB_PATH) $(LDFLAGS) $(LIBS)# Gets rid of all generated files.
clean:rm -rf $(MAKEFILE_DIR)/gen# Gets rid of target files only, leaving the host alone. Also leaves the lib
# directory untouched deliberately, so we can persist multiple architectures
# across builds for iOS and Android.
cleantarget:rm -rf $(OBJDIR)rm -rf $(BINDIR)$(DEPDIR)/%.d: ;
.PRECIOUS: $(DEPDIR)/%.d-include $(patsubst %,$(DEPDIR)/%.d,$(basename $(TF_CC_SRCS)))
修改完成后再次执行 ./tensorflow/contrib/lite/build_rpi_lib.sh
,此时在 ./tensorflow/contrib/lite/gen/bin/rpi_armv8
目录下会产生编译好的label_image二进制文件。
5.拷贝程序到板子上
准备测试图片 tensorflow/contrib/lite/examples/label_image/testdata/grace_hopper.bmp
,当然用其他的图片测试也行。此外,还需要从https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/g3doc/models.md 地址下载你想要测试的tf lite模型。还需要准备ImageNet的标签文件。最后需要传到板子上的是如下几个文件。
grace_hopper.bmp
label_image
labels.txt
mobilenet_v1_1.0_224_quant.tflite/mobilenet_v1_1.0_224.tflite
6.在ARM板子上运行Tensorflow Lite
到此准备工作全部完成了,最后可以在板子上测试Tensorflow Lite的性能了,使用姿势如下:
./label_image -v 1 -m ./mobilenet_v1_1.0_224.tflite -i ./grace_hopper.bmp -l ./labels.txt
运行效果如下:
average time: 855.91 ms
0.860174: 653 military uniform
0.0481021: 907 Windsor tie
0.00786706: 466 bulletproof vest
0.00644932: 514 cornet
0.00608028: 543 drumstick
接下来可以再测试下量化后的MobileNet效果:
./label_image -v 4 -m ./mobilenet_v1_1.0_224_quant.tflite -i ./grace_hopper.bmp -l ./labels.txt
效果如下:
average time: 185.988 ms
0.427451: 653 military uniform
0.305882: 907 Windsor tie
0.0431373: 668 mortarboard
0.0313726: 458 bow tie
0.0235294: 543 drumstick
上述Demo运行时默认线程数为10,若想测试单线程效果,只需再添加-t 1参数即可。单线程在板子上的性能效果如下,从结果看TF Lite的多线程反而没起到加速的效果(2 3 4线程数量我都试过,这个一个跟TF Lite的线程优化有关,二是可能跟ARM板子上的大小核相关,单线程情况下默认使用的是大核)
./label_image -v 1 -m ./mobilenet_v1_1.0_224.tflite -i ./grace_hopper.bmp -l ./labels.txt -t 1 average time: 297.647 ms
0.860174: 653 military uniform
0.0481024: 907 Windsor tie
0.00786706: 466 bulletproof vest
0.00644934: 514 cornet
0.00608031: 543 drumstick
./label_image -v 4 -m ./mobilenet_v1_1.0_224_quant.tflite -i ./grace_hopper.bmp -l ./labels.txt -t 1 average time: 145.173 ms
0.427451: 653 military uniform
0.305882: 907 Windsor tie
0.0431373: 668 mortarboard
0.0313726: 458 bow tie
0.0235294: 543 drumstick
从性能上看,如果不考虑识别的准确率的话,TF Lite量化后的性能收益还是很不错的。在同等平台下,NCNN跑Mobilenet的耗时比TF Lite的非量化版本略快。
Reference:
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/g3doc/rpi.md
https://pastebin.com/LDEGyG46
转载请联系本人,本文原地址:https://blog.csdn.net/computerme/article/details/80345065
在ARM板子上把玩Tensorflow Lite相关推荐
- 在Android上可视化TensorFlow Lite AI结果
目录 下一步 在这里,我们完成了基于TensorFlow Lite的应用程序的构建,该应用程序使用来自ONNX Model Zoo的网络模型执行对象识别. 输出存储在一组数字数组中.除非我们做更多的工 ...
- 基于TensorFlow Lite的人声识别在端上的实现
通过TensorFlow Lite,移动终端.IoT设备可以在端上实现声音识别,这可以应用在安防.医疗监护等领域.来自阿里巴巴闲鱼技术互动组仝辉和上叶通过TensorFlow Lite实现了一套完整的 ...
- 从零开始在Windows上构建Android版的Tensorflow Lite
文章目录 第一步:获取源代码 1. 工具:Git 2. 下载代码 第二步:了解代码 第三步:工具准备 1. Git 2. NDK 3. CMake 4. Python 3 5. Patch 第四步:环 ...
- arm linux telnet 密码,arm-linux 板子上telnetd的移植
首先在内核中要支持一样东西: 在Character devices中选中Unix98 PTY support busybox中已经可以支持telnetd的命令,我用的是busybox-1.24.1版本 ...
- 在Android上将ONNX神经网络模型与TensorFlow Lite结合使用
目录 下一步 在这里,我们从预先训练的模型中制作TensorFlow Lite模型. 这是有关在Android上使用TensorFlow Lite的系列文章中的第二篇.在上一节中,设置了开发环境以使用 ...
- Google正式发布TensorFlow Lite预览版,针对移动/嵌入设备的轻量级解决方案
来源:AI科技评论 概要:日前,谷歌正式发布 TensorFlow Lite 开发者预览版,这是针对移动和嵌入式设备的轻量级解决方案. 日前,谷歌正式发布 TensorFlow Lite 开发者预览版 ...
- 【linux基础】关于ARM板子使用O3编译选项优化
前言 应领导要求需要将最初级版本的算法移植到ARM板子上,并进行优化,以期达到实时. 平台 移植前: TX2 移植后: ARM() processor : 3 model name : ARMv7 P ...
- Tensorflow Lite人体姿势跟踪功能上线:基于PosNet的实时人体姿态估计
点击我爱计算机视觉标星,更快获取CVML新技术 本文经机器之心(微信公众号:almosthuman2014)授权转载,禁止二次转载. 选自medium 作者:TensorFlow 机器之心编译 参与: ...
- (六)将样式转换模型从TensorFlow转换为TensorFlow Lite
目录 介绍 什么是TensorFlow Lite? TensorFlow Lite转换器 运行TensorFlow Lite模型 下一步 下载 g_model_BtoA_005730.zip - 12 ...
最新文章
- 判断是不是一个数组?
- 语言 蜗牛爬树问题_一年级语文《小蜗牛》课文解析及测试题,感觉最难的是口语交际题...
- 内核电源管理器已启动关机转换_Linux系统启动流程
- 云起智慧中心连接华为_【转发】华为智慧屏HiLink控制联动,操作指南来了!
- 2017.10.17 蜘蛛难题 思考记录
- c++ qml 数组_【QML与C++混合编程】用QVariantList传递数组类型成员
- Python版本的查看
- 安装visio2019Pro提示报错“0xC004F017“具体解决办法
- uniapp 安卓/ios 录音授权,录制音频,录音文件上传
- Android GPS学习笔记—HAL实现
- JS正则表达式语法大全(非常详细)
- 恒凯USB数据采集卡在国产麒麟操作系统下的应用(二)Python环境设置及数据采集实验
- SDU程序设计思维Week15-作业 字符串
- ffmpeg压制视频命令及常用转码参数解析
- D. Three Religions
- 【汇总】ThinkPHP6入门手册
- 以 Hop、Connext 和 Nomad 为例:区块链桥如何优化资金利用效率?
- 智能体脂秤方案——什么是体脂?
- python宿舍管理保存_学生宿舍管理信息系统详细报告
- 时延、发送时延、传输时延、处理时延、排队时延、时延带宽积