Ubuntu16.04+cuda10.0+cudnn7.6配置tensorflow1.x环境
2024-04-11 14:28:18
现阶段有多种深度学习框架，应用较多的主要是tensorflow、pytorch以及百度公司的paddlepaddle。个人刚开始使用的深度学习框架是基于pytorch的mmdetection，来自于商汤科技公司，学习和使用了接近一年，利用其跑工程，学习网络等，主要是目标检测方向的应用，整体有较好把握，收获较大。近期，老板事物不多，个人时间较充裕，于是开始学习使用tensorflow，进一步了解其他的深度学习框架。下面分享一下我的环境安装过程：
tensorflow的环境配置不比mmdetection简单，但是由于有了后者的配环境经验，所以摸索起来即使遇到了困难，也能凭经验一步一步解决。我的系统是Ubuntu16.04，之前就装好了cuda，cudatoolkit和cudnn都已经版本对应了，所以少了一些前期的准备工作。
但是实际安装tensorflow的时候还是遇到了很多困难，原先计划安装tensorflow2.x版本，也从tensorflow的官网去找一些官方的安装教程，想避免走一些弯路，但最后下来，我发现我按照官网的教程倒是走了不少弯路，而且还没有成功。按照官网的教程cuda10.0是可以安装2.x版本的，但个人实践经验后还是觉得不靠谱，也和自己不熟悉有关吧，比如docker、bazel构建tensorflow等，操作起来不方便，而且也没有必要，官网把这些放在一起是真的浪费我的时间。
我开始装的是2.0版本，开始按照官网教程来的，有些指令执行不了，换方式也没解决。最后放弃了官方教程，从网上找了一些和自己环境差不多的博客进行参考，指令倒是都顺利装完了，但是跑实例的时候跑不出来，试着修改问题，但找不到源码位置，所以这次的尝试又夭折了。我后面把出现的问题找了一下：
(tf2) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf2/tensorflow/workspace/traning_demo# python model_main_tf2.py --model_dir=models/my_ssd_mobilenet_v2 --pipeline_config_path=models/my_ssd_mobilenet_v2/pipeline.config
WARNING:tensorflow:There are non-GPU devices in `tf.distribute.Strategy`, not using nccl allreduce.
W0408 14:14:37.727346 140687491278656 cross_device_ops.py:1321] There are non-GPU devices in `tf.distribute.Strategy`, not using nccl allreduce.
INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:CPU:0',)
I0408 14:14:37.727849 140687491278656 mirrored_strategy.py:350] Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:CPU:0',)
INFO:tensorflow:Maybe overwriting train_steps: None
I0408 14:14:37.742078 140687491278656 config_util.py:552] Maybe overwriting train_steps: None
INFO:tensorflow:Maybe overwriting use_bfloat16: False
I0408 14:14:37.742265 140687491278656 config_util.py:552] Maybe overwriting use_bfloat16: False
WARNING:tensorflow:From /root/anaconda3/envs/tf2/lib/python3.7/site-packages/object_detection/model_lib_v2.py:546: StrategyBase.experimental_distribute_datasets_from_function (from tensorflow.python.distribute.distribute_lib) is deprecated and will be removed in a future version.
Instructions for updating:
rename to distribute_datasets_from_function
W0408 14:14:37.806042 140687491278656 deprecation.py:339] From /root/anaconda3/envs/tf2/lib/python3.7/site-packages/object_detection/model_lib_v2.py:546: StrategyBase.experimental_distribute_datasets_from_function (from tensorflow.python.distribute.distribute_lib) is deprecated and will be removed in a future version.
Instructions for updating:
rename to distribute_datasets_from_function
INFO:tensorflow:Reading unweighted datasets: ['images/train.record']
I0408 14:14:37.813343 140687491278656 dataset_builder.py:163] Reading unweighted datasets: ['images/train.record']
INFO:tensorflow:Reading record datasets for input file: ['images/train.record']
I0408 14:14:37.814830 140687491278656 dataset_builder.py:80] Reading record datasets for input file: ['images/train.record']
INFO:tensorflow:Number of filenames to read: 1
I0408 14:14:37.814923 140687491278656 dataset_builder.py:81] Number of filenames to read: 1
WARNING:tensorflow:num_readers has been reduced to 1 to match input file shards.
W0408 14:14:37.814991 140687491278656 dataset_builder.py:88] num_readers has been reduced to 1 to match input file shards.
WARNING:tensorflow:From /root/anaconda3/envs/tf2/lib/python3.7/site-packages/object_detection/builders/dataset_builder.py:105: parallel_interleave (from tensorflow.python.data.experimental.ops.interleave_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.interleave(map_func, cycle_length, block_length, num_parallel_calls=tf.data.AUTOTUNE)` instead. If sloppy execution is desired, use `tf.data.Options.experimental_deterministic`.
W0408 14:14:37.817644 140687491278656 deprecation.py:339] From /root/anaconda3/envs/tf2/lib/python3.7/site-packages/object_detection/builders/dataset_builder.py:105: parallel_interleave (from tensorflow.python.data.experimental.ops.interleave_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.interleave(map_func, cycle_length, block_length, num_parallel_calls=tf.data.AUTOTUNE)` instead. If sloppy execution is desired, use `tf.data.Options.experimental_deterministic`.
WARNING:tensorflow:From /root/anaconda3/envs/tf2/lib/python3.7/site-packages/object_detection/builders/dataset_builder.py:237: DatasetV1.map_with_legacy_function (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.map()
W0408 14:14:37.847065 140687491278656 deprecation.py:339] From /root/anaconda3/envs/tf2/lib/python3.7/site-packages/object_detection/builders/dataset_builder.py:237: DatasetV1.map_with_legacy_function (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.map()
WARNING:tensorflow:From /root/anaconda3/envs/tf2/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py:201: sparse_to_dense (from tensorflow.python.ops.sparse_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Create a `tf.sparse.SparseTensor` and use `tf.sparse.to_dense` instead.
W0408 14:14:43.827106 140687491278656 deprecation.py:339] From /root/anaconda3/envs/tf2/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py:201: sparse_to_dense (from tensorflow.python.ops.sparse_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Create a `tf.sparse.SparseTensor` and use `tf.sparse.to_dense` instead.
WARNING:tensorflow:From /root/anaconda3/envs/tf2/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py:201: sample_distorted_bounding_box (from tensorflow.python.ops.image_ops_impl) is deprecated and will be removed in a future version.
Instructions for updating:
`seed2` arg is deprecated.Use sample_distorted_bounding_box_v2 instead.
W0408 14:14:47.471886 140687491278656 deprecation.py:339] From /root/anaconda3/envs/tf2/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py:201: sample_distorted_bounding_box (from tensorflow.python.ops.image_ops_impl) is deprecated and will be removed in a future version.
Instructions for updating:
`seed2` arg is deprecated.Use sample_distorted_bounding_box_v2 instead.
WARNING:tensorflow:From /root/anaconda3/envs/tf2/lib/python3.7/site-packages/object_detection/inputs.py:282: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
W0408 14:14:49.574233 140687491278656 deprecation.py:339] From /root/anaconda3/envs/tf2/lib/python3.7/site-packages/object_detection/inputs.py:282: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
/root/anaconda3/envs/tf2/lib/python3.7/site-packages/tensorflow/python/keras/backend.py:434: UserWarning: `tf.keras.backend.set_learning_phase` is deprecated and will be removed after 2020-10-11. To update it, simply pass a True/False value to the `training` argument of the `__call__` method of your layer or model.warnings.warn('`tf.keras.backend.set_learning_phase` is deprecated and '
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 14:15:02.898844 140671520339712 convolutional_keras_box_predictor.py:154] depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 14:15:02.899355 140671520339712 convolutional_keras_box_predictor.py:154] depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 14:15:02.899594 140671520339712 convolutional_keras_box_predictor.py:154] depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 14:15:02.899832 140671520339712 convolutional_keras_box_predictor.py:154] depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 14:15:02.900039 140671520339712 convolutional_keras_box_predictor.py:154] depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 14:15:02.900240 140671520339712 convolutional_keras_box_predictor.py:154] depth of additional conv before box predictor: 0
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._groundtruth_lists
W0408 14:15:23.303248 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._groundtruth_lists
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor
W0408 14:15:23.303692 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._batched_prediction_tensor_names
W0408 14:15:23.303777 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._batched_prediction_tensor_names
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads
W0408 14:15:23.303838 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._sorted_head_names
W0408 14:15:23.303881 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._sorted_head_names
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._shared_nets
W0408 14:15:23.303931 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._shared_nets
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings
W0408 14:15:23.303973 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background
W0408 14:15:23.304014 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._shared_nets.0
W0408 14:15:23.304055 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._shared_nets.0
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._shared_nets.1
W0408 14:15:23.304095 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._shared_nets.1
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._shared_nets.2
W0408 14:15:23.304135 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._shared_nets.2
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._shared_nets.3
W0408 14:15:23.304175 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._shared_nets.3
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._shared_nets.4
W0408 14:15:23.304216 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._shared_nets.4
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._shared_nets.5
W0408 14:15:23.304257 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._shared_nets.5
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.0
W0408 14:15:23.304313 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.0
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.1
W0408 14:15:23.304363 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.1
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.2
W0408 14:15:23.304403 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.2
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.3
W0408 14:15:23.304442 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.3
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.4
W0408 14:15:23.304482 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.4
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.5
W0408 14:15:23.304522 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.5
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.0
W0408 14:15:23.304562 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.0
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.1
W0408 14:15:23.304601 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.1
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.2
W0408 14:15:23.304653 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.2
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.3
W0408 14:15:23.304707 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.3
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.4
W0408 14:15:23.304747 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.4
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.5
W0408 14:15:23.304787 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.5
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.0._box_encoder_layers
W0408 14:15:23.304844 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.0._box_encoder_layers
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.1._box_encoder_layers
W0408 14:15:23.304888 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.1._box_encoder_layers
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.2._box_encoder_layers
W0408 14:15:23.304928 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.2._box_encoder_layers
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.3._box_encoder_layers
W0408 14:15:23.304969 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.3._box_encoder_layers
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.4._box_encoder_layers
W0408 14:15:23.305009 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.4._box_encoder_layers
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.5._box_encoder_layers
W0408 14:15:23.305050 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.5._box_encoder_layers
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.0._class_predictor_layers
W0408 14:15:23.305090 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.0._class_predictor_layers
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.1._class_predictor_layers
W0408 14:15:23.305131 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.1._class_predictor_layers
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.2._class_predictor_layers
W0408 14:15:23.305172 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.2._class_predictor_layers
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.3._class_predictor_layers
W0408 14:15:23.305213 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.3._class_predictor_layers
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.4._class_predictor_layers
W0408 14:15:23.305254 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.4._class_predictor_layers
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.5._class_predictor_layers
W0408 14:15:23.305295 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.5._class_predictor_layers
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.0._box_encoder_layers.0
W0408 14:15:23.305337 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.0._box_encoder_layers.0
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.1._box_encoder_layers.0
W0408 14:15:23.305378 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.1._box_encoder_layers.0
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.2._box_encoder_layers.0
W0408 14:15:23.305418 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.2._box_encoder_layers.0
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.3._box_encoder_layers.0
W0408 14:15:23.305459 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.3._box_encoder_layers.0
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.4._box_encoder_layers.0
W0408 14:15:23.305498 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.4._box_encoder_layers.0
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.5._box_encoder_layers.0
W0408 14:15:23.305538 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.5._box_encoder_layers.0
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.0._class_predictor_layers.0
W0408 14:15:23.305578 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.0._class_predictor_layers.0
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.1._class_predictor_layers.0
W0408 14:15:23.305619 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.1._class_predictor_layers.0
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.2._class_predictor_layers.0
W0408 14:15:23.305659 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.2._class_predictor_layers.0
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.3._class_predictor_layers.0
W0408 14:15:23.305704 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.3._class_predictor_layers.0
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.4._class_predictor_layers.0
W0408 14:15:23.305745 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.4._class_predictor_layers.0
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.5._class_predictor_layers.0
W0408 14:15:23.305785 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.5._class_predictor_layers.0
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.0._box_encoder_layers.0.kernel
W0408 14:15:23.305828 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.0._box_encoder_layers.0.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.0._box_encoder_layers.0.bias
W0408 14:15:23.305869 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.0._box_encoder_layers.0.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.1._box_encoder_layers.0.kernel
W0408 14:15:23.305909 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.1._box_encoder_layers.0.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.1._box_encoder_layers.0.bias
W0408 14:15:23.305949 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.1._box_encoder_layers.0.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.2._box_encoder_layers.0.kernel
W0408 14:15:23.305990 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.2._box_encoder_layers.0.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.2._box_encoder_layers.0.bias
W0408 14:15:23.306030 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.2._box_encoder_layers.0.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.3._box_encoder_layers.0.kernel
W0408 14:15:23.306070 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.3._box_encoder_layers.0.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.3._box_encoder_layers.0.bias
W0408 14:15:23.306112 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.3._box_encoder_layers.0.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.4._box_encoder_layers.0.kernel
W0408 14:15:23.306151 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.4._box_encoder_layers.0.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.4._box_encoder_layers.0.bias
W0408 14:15:23.306191 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.4._box_encoder_layers.0.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.5._box_encoder_layers.0.kernel
W0408 14:15:23.306232 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.5._box_encoder_layers.0.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.5._box_encoder_layers.0.bias
W0408 14:15:23.306275 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.box_encodings.5._box_encoder_layers.0.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.0._class_predictor_layers.0.kernel
W0408 14:15:23.306315 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.0._class_predictor_layers.0.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.0._class_predictor_layers.0.bias
W0408 14:15:23.306356 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.0._class_predictor_layers.0.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.1._class_predictor_layers.0.kernel
W0408 14:15:23.306397 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.1._class_predictor_layers.0.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.1._class_predictor_layers.0.bias
W0408 14:15:23.306438 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.1._class_predictor_layers.0.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.2._class_predictor_layers.0.kernel
W0408 14:15:23.306478 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.2._class_predictor_layers.0.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.2._class_predictor_layers.0.bias
W0408 14:15:23.306519 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.2._class_predictor_layers.0.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.3._class_predictor_layers.0.kernel
W0408 14:15:23.306560 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.3._class_predictor_layers.0.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.3._class_predictor_layers.0.bias
W0408 14:15:23.306600 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.3._class_predictor_layers.0.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.4._class_predictor_layers.0.kernel
W0408 14:15:23.306641 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.4._class_predictor_layers.0.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.4._class_predictor_layers.0.bias
W0408 14:15:23.306685 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.4._class_predictor_layers.0.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.5._class_predictor_layers.0.kernel
W0408 14:15:23.306726 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.5._class_predictor_layers.0.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.5._class_predictor_layers.0.bias
W0408 14:15:23.306768 140687491278656 util.py:161] Unresolved object in checkpoint: (root).model._box_predictor._prediction_heads.class_predictions_with_background.5._class_predictor_layers.0.bias
WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details.
W0408 14:15:23.306821 140687491278656 util.py:169] A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details.
WARNING:tensorflow:From /root/anaconda3/envs/tf2/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py:605: calling map_fn_v2 (from tensorflow.python.ops.map_fn) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Use fn_output_signature instead
W0408 14:15:32.760873 140644878092032 deprecation.py:537] From /root/anaconda3/envs/tf2/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py:605: calling map_fn_v2 (from tensorflow.python.ops.map_fn) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Use fn_output_signature instead最后就这么卡住了，看开始报的问题，还无法调用GPU,后面我查看工程文件，还是增加了一些训练的内容，也就是环境是搭上了，但并不对。我也怀疑是版本的问题，于是下一步改为装1.x版本。
针对无法调用gpu这个问题，我上网搜了我报错的地方，发现没什么有效的解决方法，我在执行完安装tensorflow的命令后验证是否可以使用GPU上找到了一点思路，我不按官网说的linux版本使用pip install tensorflow来安装，我直接装我这个版本可以安装的tensorflow-gpu，再去执行验证指令，发现可以调用GPU，我再下一步安装对应版本的其他配置文件。这是我使用的验证指令：
import tensorflow as tftf.test.is_gpu_available()
发现返回的是true 说明可以调用GPU了
装完之后，我的环境是这样的：
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/workspace/training_demo# conda list
# packages in environment at /root/anaconda3/envs/tf1:
#
# Name Version Build Channel
_libgcc_mutex 0.1 main
_tflow_select 2.1.0 gpu
absl-py 0.12.0 py36h06a4308_0
astor 0.8.1 py36h06a4308_0
blas 1.0 mkl
c-ares 1.17.1 h27cfd23_0
ca-certificates 2021.1.19 h06a4308_1
certifi 2020.12.5 py36h06a4308_0
contextlib2 0.6.0.post1 pypi_0 pypi
coverage 5.5 py36h27cfd23_2
cudatoolkit 10.0.130 0
cudnn 7.6.5 cuda10.0_0
cupti 10.0.130 0
cycler 0.10.0 pypi_0 pypi
cython 0.29.22 py36h2531618_0
gast 0.2.2 py36_0
google-pasta 0.2.0 py_0
grpcio 1.36.1 py36h2157cd5_1
h5py 2.10.0 py36hd6299e0_1
hdf5 1.10.6 hb1b8bf9_0
importlib-metadata 3.7.3 py36h06a4308_1
intel-openmp 2020.2 254
keras-applications 1.0.8 py_1
keras-preprocessing 1.1.2 pyhd3eb1b0_0
kiwisolver 1.3.1 pypi_0 pypi
ld_impl_linux-64 2.33.1 h53a641e_7
libffi 3.3 he6710b0_2
libgcc-ng 9.1.0 hdf63c60_0
libgfortran-ng 7.3.0 hdf63c60_0
libprotobuf 3.14.0 h8c45485_0
libstdcxx-ng 9.1.0 hdf63c60_0
lvis 0.5.3 pypi_0 pypi
lxml 4.6.3 pypi_0 pypi
markdown 3.3.4 py36h06a4308_0
matplotlib 3.3.4 pypi_0 pypi
mkl 2020.2 256
mkl-service 2.3.0 py36he8ac12f_0
mkl_fft 1.3.0 py36h54f3939_0
mkl_random 1.1.1 py36h0573a6f_0
ncurses 6.2 he6710b0_1
numpy 1.19.2 py36h54aff64_0
numpy-base 1.19.2 py36hfa32c7d_0
object-detection 0.1 pypi_0 pypi
opencv-python 4.5.1.48 pypi_0 pypi
openssl 1.1.1k h27cfd23_0
opt_einsum 3.1.0 py_0
pandas 1.1.5 pypi_0 pypi
pillow 8.2.0 pypi_0 pypi
pip 21.0.1 py36h06a4308_0
protobuf 3.14.0 py36h2531618_1
pycocotools 2.0.2 pypi_0 pypi
pyparsing 2.4.7 pypi_0 pypi
python 3.6.13 hdb3f193_0
python-dateutil 2.8.1 pypi_0 pypi
pytz 2021.1 pypi_0 pypi
readline 8.1 h27cfd23_0
scipy 1.5.2 py36h0b6359f_0
setuptools 52.0.0 py36h06a4308_0
six 1.15.0 py36h06a4308_0
sqlite 3.35.4 hdfb4753_0
tensorboard 1.15.0 pyhb230dea_0
tensorflow 1.15.0 gpu_py36h5a509aa_0
tensorflow-base 1.15.0 gpu_py36h9dcbed7_0
tensorflow-estimator 1.15.1 pyh2649769_0
tensorflow-gpu 1.15.0 h0d30ee6_0
termcolor 1.1.0 py36h06a4308_1
tf-slim 1.1.0 pypi_0 pypi
tk 8.6.10 hbc83047_0
typing_extensions 3.7.4.3 pyha847dfd_0
webencodings 0.5.1 py36_1
werkzeug 0.16.1 py_0
wheel 0.36.2 pyhd3eb1b0_0
wrapt 1.12.1 py36h7b6447c_1
xz 5.2.5 h7b6447c_0
zipp 3.4.1 pyhd3eb1b0_0
zlib 1.2.11 h7b6447c_3 我也把我的安装过程大致整理了一下：
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1# conda install protobuf
Collecting package metadata (current_repodata.json): done
Solving environment: done# All requested packages already installed.(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1# cd tensorflow/
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow# mkdir workspace
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow# cd workspace/
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/workspace# mkdir training_demo
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/workspace# cd training_demo/
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/workspace/training_demo# chmod -R a=rwx ./
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/workspace/training_demo# mkdir annotations
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/workspace/training_demo# mkdir exported-models
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/workspace/training_demo# mkdir images
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/workspace/training_demo# mkdir models
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/workspace/training_demo# mkdir pretrained-models
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/workspace/training_demo# chmod -R a=rwx ./
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/workspace/training_demo# cd ../..
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow# cd models/research/
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/models/research# protoc object_detection/protos/*.proto --python_out=.
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/models/research# git clone https://github.com/cocodataset/cocoapi.git
Cloning into 'cocoapi'...
remote: Enumerating objects: 975, done.
remote: Total 975 (delta 0), reused 0 (delta 0), pack-reused 975
Receiving objects: 100% (975/975), 11.72 MiB | 5.96 MiB/s, done.
Resolving deltas: 100% (576/576), done.
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/models/research# cd cocoapi/PythonAPI/
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/models/research/cocoapi/PythonAPI# make
python setup.py build_ext --inplace
running build_ext
cythoning pycocotools/_mask.pyx to pycocotools/_mask.c
/root/anaconda3/envs/tf1/lib/python3.6/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /nfs/private/junnxie/tf1/tensorflow/models/research/cocoapi/PythonAPI/pycocotools/_mask.pyxtree = Parsing.p_module(s, pxd, full_module_name)
building 'pycocotools._mask' extension
creating build
creating build/common
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/pycocotools
gcc -pthread -B /root/anaconda3/envs/tf1/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/root/.local/lib/python3.6/site-packages/numpy/core/include -I../common -I/root/anaconda3/envs/tf1/include/python3.6m -c ../common/maskApi.c -o build/temp.linux-x86_64-3.6/../common/maskApi.o -Wno-cpp -Wno-unused-function -std=c99
../common/maskApi.c: In function ‘rleDecode’:
../common/maskApi.c:46:7: warning: this ‘for’ clause does not guard... [-Wmisleading-indentation]for( k=0; k<R[i].cnts[j]; k++ ) *(M++)=v; v=!v; }}^~~
../common/maskApi.c:46:49: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the ‘for’for( k=0; k<R[i].cnts[j]; k++ ) *(M++)=v; v=!v; }}^
../common/maskApi.c: In function ‘rleFrPoly’:
../common/maskApi.c:166:3: warning: this ‘for’ clause does not guard... [-Wmisleading-indentation]for(j=0; j<k; j++) x[j]=(int)(scale*xy[j*2+0]+.5); x[k]=x[0];^~~
../common/maskApi.c:166:54: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the ‘for’for(j=0; j<k; j++) x[j]=(int)(scale*xy[j*2+0]+.5); x[k]=x[0];^
../common/maskApi.c:167:3: warning: this ‘for’ clause does not guard... [-Wmisleading-indentation]for(j=0; j<k; j++) y[j]=(int)(scale*xy[j*2+1]+.5); y[k]=y[0];^~~
../common/maskApi.c:167:54: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the ‘for’for(j=0; j<k; j++) y[j]=(int)(scale*xy[j*2+1]+.5); y[k]=y[0];^
../common/maskApi.c: In function ‘rleToString’:
../common/maskApi.c:212:7: warning: this ‘if’ clause does not guard... [-Wmisleading-indentation]if(more) c |= 0x20; c+=48; s[p++]=c;^~
../common/maskApi.c:212:27: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the ‘if’if(more) c |= 0x20; c+=48; s[p++]=c;^
../common/maskApi.c: In function ‘rleFrString’:
../common/maskApi.c:220:3: warning: this ‘while’ clause does not guard... [-Wmisleading-indentation]while( s[m] ) m++; cnts=malloc(sizeof(uint)*m); m=0;^~~~~
../common/maskApi.c:220:22: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the ‘while’while( s[m] ) m++; cnts=malloc(sizeof(uint)*m); m=0;^~~~
../common/maskApi.c:228:5: warning: this ‘if’ clause does not guard... [-Wmisleading-indentation]if(m>2) x+=(long) cnts[m-2]; cnts[m++]=(uint) x;^~
../common/maskApi.c:228:34: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the ‘if’if(m>2) x+=(long) cnts[m-2]; cnts[m++]=(uint) x;^~~~
../common/maskApi.c: In function ‘rleToBbox’:
../common/maskApi.c:141:31: warning: ‘xp’ may be used uninitialized in this function [-Wmaybe-uninitialized]if(j%2==0) xp=x; else if(xp<x) { ys=0; ye=h-1; }^
gcc -pthread -B /root/anaconda3/envs/tf1/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/root/.local/lib/python3.6/site-packages/numpy/core/include -I../common -I/root/anaconda3/envs/tf1/include/python3.6m -c pycocotools/_mask.c -o build/temp.linux-x86_64-3.6/pycocotools/_mask.o -Wno-cpp -Wno-unused-function -std=c99
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/pycocotools
gcc -pthread -shared -B /root/anaconda3/envs/tf1/compiler_compat -L/root/anaconda3/envs/tf1/lib -Wl,-rpath=/root/anaconda3/envs/tf1/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.6/../common/maskApi.o build/temp.linux-x86_64-3.6/pycocotools/_mask.o -o build/lib.linux-x86_64-3.6/pycocotools/_mask.cpython-36m-x86_64-linux-gnu.so
copying build/lib.linux-x86_64-3.6/pycocotools/_mask.cpython-36m-x86_64-linux-gnu.so -> pycocotools
rm -rf build
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/models/research/cocoapi/PythonAPI# cp -r pycocotools /nfs/private/junnxie/tf1/tensorflow/models/research/
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/models/research/cocoapi/PythonAPI# cd ../..
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/models/research# cp object_detection/packages/tf1/setup.py .
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/models/research# python
Python 3.6.13 |Anaconda, Inc.| (default, Feb 23 2021, 21:15:04)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>> tf.__version__
'1.15.0'
>>>
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/models/research# python -m pip install .
Processing /nfs/private/junnxie/tf1/tensorflow/models/research
Collecting pillowDownloading Pillow-8.2.0-cp36-cp36m-manylinux1_x86_64.whl (3.0 MB)|????????????????????????????????| 3.0 MB 1.9 MB/s
Collecting lxmlDownloading lxml-4.6.3-cp36-cp36m-manylinux1_x86_64.whl (5.5 MB)|????????????????????????????????| 5.5 MB 18.7 MB/s
Collecting matplotlibUsing cached matplotlib-3.3.4-cp36-cp36m-manylinux1_x86_64.whl (11.5 MB)
Requirement already satisfied: Cython in /root/anaconda3/envs/tf1/lib/python3.6/site-packages (from object-detection==0.1) (0.29.22)
Collecting contextlib2Using cached contextlib2-0.6.0.post1-py2.py3-none-any.whl (9.8 kB)
Collecting tf-slimUsing cached tf_slim-1.1.0-py2.py3-none-any.whl (352 kB)
Requirement already satisfied: six in /root/anaconda3/envs/tf1/lib/python3.6/site-packages (from object-detection==0.1) (1.15.0)
Collecting pycocotoolsUsing cached pycocotools-2.0.2.tar.gz (23 kB)
Collecting lvisUsing cached lvis-0.5.3-py3-none-any.whl (14 kB)
Requirement already satisfied: scipy in /root/anaconda3/envs/tf1/lib/python3.6/site-packages (from object-detection==0.1) (1.5.2)
Collecting pandasDownloading pandas-1.1.5-cp36-cp36m-manylinux1_x86_64.whl (9.5 MB)|????????????????????????????????| 9.5 MB 25.6 MB/s
Collecting cycler>=0.10.0Using cached cycler-0.10.0-py2.py3-none-any.whl (6.5 kB)
Collecting pyparsing>=2.4.0Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting opencv-python>=4.1.0.25Using cached opencv_python-4.5.1.48-cp36-cp36m-manylinux2014_x86_64.whl (50.4 MB)
Requirement already satisfied: numpy>=1.18.2 in /root/.local/lib/python3.6/site-packages (from lvis->object-detection==0.1) (1.19.5)
Collecting kiwisolver>=1.1.0Using cached kiwisolver-1.3.1-cp36-cp36m-manylinux1_x86_64.whl (1.1 MB)
Collecting python-dateutil>=2.8.0Using cached python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
Collecting pytz>=2017.2Using cached pytz-2021.1-py2.py3-none-any.whl (510 kB)
Requirement already satisfied: setuptools>=18.0 in /root/anaconda3/envs/tf1/lib/python3.6/site-packages (from pycocotools->object-detection==0.1) (52.0.0.post20210125)
Requirement already satisfied: absl-py>=0.2.2 in /root/anaconda3/envs/tf1/lib/python3.6/site-packages (from tf-slim->object-detection==0.1) (0.12.0)
Building wheels for collected packages: object-detection, pycocotoolsBuilding wheel for object-detection (setup.py) ... doneCreated wheel for object-detection: filename=object_detection-0.1-py3-none-any.whl size=1643984 sha256=89a5fc6e88577caab4206a9a4a200eb8dba4a7e5c2b970ff9637ad300854d029Stored in directory: /tmp/pip-ephem-wheel-cache-zp4g9lh_/wheels/22/22/a1/fc116dd4526d44ac96cbfb85fb89f0a9c5cb88b825a6c25020Building wheel for pycocotools (setup.py) ... doneCreated wheel for pycocotools: filename=pycocotools-2.0.2-cp36-cp36m-linux_x86_64.whl size=273408 sha256=b9eb3f09d6addcd01d7c90fde11e61852ca1dd913c31022c5d2ec1f016a600aaStored in directory: /root/.cache/pip/wheels/d8/c2/ba/8f5306f921c2e79ad7b09effdfed6bd966cfcf8c6fe55422d6
Successfully built object-detection pycocotools
Installing collected packages: python-dateutil, pyparsing, pillow, kiwisolver, cycler, pytz, opencv-python, matplotlib, tf-slim, pycocotools, pandas, lxml, lvis, contextlib2, object-detection
Successfully installed contextlib2-0.6.0.post1 cycler-0.10.0 kiwisolver-1.3.1 lvis-0.5.3 lxml-4.6.3 matplotlib-3.3.4 object-detection-0.1 opencv-python-4.5.1.48 pandas-1.1.5 pillow-8.2.0 pycocotools-2.0.2 pyparsing-2.4.7 python-dateutil-2.8.1 pytz-2021.1 tf-slim-1.1.0
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/models/research# python object_detection/builders/model_builder_tf1_test.py
Running tests under Python 3.6.13: /root/anaconda3/envs/tf1/bin/python
[ RUN ] ModelBuilderTF1Test.test_create_context_rcnn_from_config_with_params0 (True)
[ OK ] ModelBuilderTF1Test.test_create_context_rcnn_from_config_with_params0 (True)
[ RUN ] ModelBuilderTF1Test.test_create_context_rcnn_from_config_with_params1 (False)
[ OK ] ModelBuilderTF1Test.test_create_context_rcnn_from_config_with_params1 (False)
[ RUN ] ModelBuilderTF1Test.test_create_experimental_model
[ OK ] ModelBuilderTF1Test.test_create_experimental_model
[ RUN ] ModelBuilderTF1Test.test_create_faster_rcnn_from_config_with_crop_feature0 (True)
[ OK ] ModelBuilderTF1Test.test_create_faster_rcnn_from_config_with_crop_feature0 (True)
[ RUN ] ModelBuilderTF1Test.test_create_faster_rcnn_from_config_with_crop_feature1 (False)
[ OK ] ModelBuilderTF1Test.test_create_faster_rcnn_from_config_with_crop_feature1 (False)
[ RUN ] ModelBuilderTF1Test.test_create_faster_rcnn_model_from_config_with_example_miner
[ OK ] ModelBuilderTF1Test.test_create_faster_rcnn_model_from_config_with_example_miner
[ RUN ] ModelBuilderTF1Test.test_create_faster_rcnn_models_from_config_faster_rcnn_with_matmul
[ OK ] ModelBuilderTF1Test.test_create_faster_rcnn_models_from_config_faster_rcnn_with_matmul
[ RUN ] ModelBuilderTF1Test.test_create_faster_rcnn_models_from_config_faster_rcnn_without_matmul
[ OK ] ModelBuilderTF1Test.test_create_faster_rcnn_models_from_config_faster_rcnn_without_matmul
[ RUN ] ModelBuilderTF1Test.test_create_faster_rcnn_models_from_config_mask_rcnn_with_matmul
[ OK ] ModelBuilderTF1Test.test_create_faster_rcnn_models_from_config_mask_rcnn_with_matmul
[ RUN ] ModelBuilderTF1Test.test_create_faster_rcnn_models_from_config_mask_rcnn_without_matmul
[ OK ] ModelBuilderTF1Test.test_create_faster_rcnn_models_from_config_mask_rcnn_without_matmul
[ RUN ] ModelBuilderTF1Test.test_create_rfcn_model_from_config
[ OK ] ModelBuilderTF1Test.test_create_rfcn_model_from_config
[ RUN ] ModelBuilderTF1Test.test_create_ssd_fpn_model_from_config
[ OK ] ModelBuilderTF1Test.test_create_ssd_fpn_model_from_config
[ RUN ] ModelBuilderTF1Test.test_create_ssd_models_from_config
[ OK ] ModelBuilderTF1Test.test_create_ssd_models_from_config
[ RUN ] ModelBuilderTF1Test.test_invalid_faster_rcnn_batchnorm_update
[ OK ] ModelBuilderTF1Test.test_invalid_faster_rcnn_batchnorm_update
[ RUN ] ModelBuilderTF1Test.test_invalid_first_stage_nms_iou_threshold
[ OK ] ModelBuilderTF1Test.test_invalid_first_stage_nms_iou_threshold
[ RUN ] ModelBuilderTF1Test.test_invalid_model_config_proto
[ OK ] ModelBuilderTF1Test.test_invalid_model_config_proto
[ RUN ] ModelBuilderTF1Test.test_invalid_second_stage_batch_size
[ OK ] ModelBuilderTF1Test.test_invalid_second_stage_batch_size
[ RUN ] ModelBuilderTF1Test.test_session
[ SKIPPED ] ModelBuilderTF1Test.test_session
[ RUN ] ModelBuilderTF1Test.test_unknown_faster_rcnn_feature_extractor
[ OK ] ModelBuilderTF1Test.test_unknown_faster_rcnn_feature_extractor
[ RUN ] ModelBuilderTF1Test.test_unknown_meta_architecture
[ OK ] ModelBuilderTF1Test.test_unknown_meta_architecture
[ RUN ] ModelBuilderTF1Test.test_unknown_ssd_feature_extractor
[ OK ] ModelBuilderTF1Test.test_unknown_ssd_feature_extractor
----------------------------------------------------------------------
Ran 21 tests in 0.245sOK (skipped=1)
我把指令进一步提炼一下,主要是省去了一些中间过程和结果：
conda create -n tf1 python=3.6
conda activate tf1
conda install tensorflow-gpu=1.15.0
conda install cudatoolkit=10.0
conda install cudnn=7.6(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1# conda install protobuf(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow# cd models/research/
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/models/research# protoc object_detection/protos/*.proto --python_out=.
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/models/research# git clone https://github.com/cocodataset/cocoapi.git(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/models/research# cd cocoapi/PythonAPI/
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/models/research/cocoapi/PythonAPI# make(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/models/research/cocoapi/PythonAPI# cp -r pycocotools /nfs/private/junnxie/tf1/tensorflow/models/research/
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/models/research/cocoapi/PythonAPI# cd ../..
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/models/research# cp object_detection/packages/tf1/setup.py .
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/models/research# python
Python 3.6.13 |Anaconda, Inc.| (default, Feb 23 2021, 21:15:04)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>> tf.__version__
'1.15.0'
>>>
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/models/research# python -m pip install .(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/models/research# python object_detection/builders/model_builder_tf1_test.py 最后，我运行了一个目标检测工程，数据是网上找的，使用tensorflow提供的代码制作标准的数据集用于训练。
训练过程如下：
(tf1) root@k8s-deploy-jcjbxf-1609678570414-7fffc8595-hkrph:/nfs/private/junnxie/tf1/tensorflow/workspace/training_demo# python model_main.py --model_dir=models/my_ssd_inception_v2 --pipeline_config_path=models/my_ssd_inception_v2/pipeline.config
WARNING:tensorflow:Forced number of epochs for all eval validations to be 1.
W0408 19:36:14.746835 140553343391552 model_lib.py:813] Forced number of epochs for all eval validations to be 1.
INFO:tensorflow:Maybe overwriting train_steps: None
I0408 19:36:14.747101 140553343391552 config_util.py:552] Maybe overwriting train_steps: None
INFO:tensorflow:Maybe overwriting use_bfloat16: False
I0408 19:36:14.747195 140553343391552 config_util.py:552] Maybe overwriting use_bfloat16: False
INFO:tensorflow:Maybe overwriting sample_1_of_n_eval_examples: 1
I0408 19:36:14.747280 140553343391552 config_util.py:552] Maybe overwriting sample_1_of_n_eval_examples: 1
INFO:tensorflow:Maybe overwriting eval_num_epochs: 1
I0408 19:36:14.747370 140553343391552 config_util.py:552] Maybe overwriting eval_num_epochs: 1
WARNING:tensorflow:Expected number of evaluation epochs is 1, but instead encountered `eval_on_train_input_config.num_epochs` = 0. Overwriting `num_epochs` to 1.
W0408 19:36:14.747480 140553343391552 model_lib.py:829] Expected number of evaluation epochs is 1, but instead encountered `eval_on_train_input_config.num_epochs` = 0. Overwriting `num_epochs` to 1.
INFO:tensorflow:create_estimator_and_inputs: use_tpu False, export_to_tpu None
I0408 19:36:14.747575 140553343391552 model_lib.py:866] create_estimator_and_inputs: use_tpu False, export_to_tpu None
INFO:tensorflow:Using config: {'_model_dir': 'models/my_ssd_inception_v2', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {rewrite_options {meta_optimizer_iterations: ONE}
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7fd4ed457e10>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
I0408 19:36:14.748096 140553343391552 estimator.py:212] Using config: {'_model_dir': 'models/my_ssd_inception_v2', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {rewrite_options {meta_optimizer_iterations: ONE}
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7fd4ed457e10>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
WARNING:tensorflow:Estimator's model_fn (<function create_model_fn.<locals>.model_fn at 0x7fd4ed462378>) includes params argument, but params are not passed to Estimator.
W0408 19:36:14.749072 140553343391552 model_fn.py:630] Estimator's model_fn (<function create_model_fn.<locals>.model_fn at 0x7fd4ed462378>) includes params argument, but params are not passed to Estimator.
INFO:tensorflow:Not using Distribute Coordinator.
I0408 19:36:14.749960 140553343391552 estimator_training.py:186] Not using Distribute Coordinator.
INFO:tensorflow:Running training and evaluation locally (non-distributed).
I0408 19:36:14.750178 140553343391552 training.py:612] Running training and evaluation locally (non-distributed).
INFO:tensorflow:Start train and evaluate loop. The evaluate will happen after every checkpoint. Checkpoint frequency is determined based on RunConfig arguments: save_checkpoints_steps None or save_checkpoints_secs 600.
I0408 19:36:14.750450 140553343391552 training.py:700] Start train and evaluate loop. The evaluate will happen after every checkpoint. Checkpoint frequency is determined based on RunConfig arguments: save_checkpoints_steps None or save_checkpoints_secs 600.
WARNING:tensorflow:From /root/anaconda3/envs/tf1/lib/python3.6/site-packages/tensorflow_core/python/training/training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
W0408 19:36:14.780125 140553343391552 deprecation.py:323] From /root/anaconda3/envs/tf1/lib/python3.6/site-packages/tensorflow_core/python/training/training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
INFO:tensorflow:Reading unweighted datasets: ['annotations/train.record']
I0408 19:36:14.899453 140553343391552 dataset_builder.py:163] Reading unweighted datasets: ['annotations/train.record']
INFO:tensorflow:Reading record datasets for input file: ['annotations/train.record']
I0408 19:36:14.906244 140553343391552 dataset_builder.py:80] Reading record datasets for input file: ['annotations/train.record']
INFO:tensorflow:Number of filenames to read: 1
I0408 19:36:14.906542 140553343391552 dataset_builder.py:81] Number of filenames to read: 1
WARNING:tensorflow:num_readers has been reduced to 1 to match input file shards.
W0408 19:36:14.906724 140553343391552 dataset_builder.py:88] num_readers has been reduced to 1 to match input file shards.
WARNING:tensorflow:From /root/anaconda3/envs/tf1/lib/python3.6/site-packages/object_detection/builders/dataset_builder.py:105: parallel_interleave (from tensorflow.python.data.experimental.ops.interleave_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.interleave(map_func, cycle_length, block_length, num_parallel_calls=tf.data.experimental.AUTOTUNE)` instead. If sloppy execution is desired, use `tf.data.Options.experimental_determinstic`.
W0408 19:36:14.920520 140553343391552 deprecation.py:323] From /root/anaconda3/envs/tf1/lib/python3.6/site-packages/object_detection/builders/dataset_builder.py:105: parallel_interleave (from tensorflow.python.data.experimental.ops.interleave_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.interleave(map_func, cycle_length, block_length, num_parallel_calls=tf.data.experimental.AUTOTUNE)` instead. If sloppy execution is desired, use `tf.data.Options.experimental_determinstic`.
WARNING:tensorflow:From /root/anaconda3/envs/tf1/lib/python3.6/site-packages/object_detection/builders/dataset_builder.py:237: DatasetV1.map_with_legacy_function (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.map()
W0408 19:36:14.985749 140553343391552 deprecation.py:323] From /root/anaconda3/envs/tf1/lib/python3.6/site-packages/object_detection/builders/dataset_builder.py:237: DatasetV1.map_with_legacy_function (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.map()
WARNING:tensorflow:From /root/anaconda3/envs/tf1/lib/python3.6/site-packages/object_detection/inputs.py:110: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
W0408 19:36:27.862425 140553343391552 deprecation.py:323] From /root/anaconda3/envs/tf1/lib/python3.6/site-packages/object_detection/inputs.py:110: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
WARNING:tensorflow:From /root/anaconda3/envs/tf1/lib/python3.6/site-packages/object_detection/inputs.py:94: sparse_to_dense (from tensorflow.python.ops.sparse_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Create a `tf.sparse.SparseTensor` and use `tf.sparse.to_dense` instead.
W0408 19:36:28.015421 140553343391552 deprecation.py:323] From /root/anaconda3/envs/tf1/lib/python3.6/site-packages/object_detection/inputs.py:94: sparse_to_dense (from tensorflow.python.ops.sparse_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Create a `tf.sparse.SparseTensor` and use `tf.sparse.to_dense` instead.
WARNING:tensorflow:From /root/anaconda3/envs/tf1/lib/python3.6/site-packages/tensorflow_core/python/autograph/operators/control_flow.py:1004: sample_distorted_bounding_box (from tensorflow.python.ops.image_ops_impl) is deprecated and will be removed in a future version.
Instructions for updating:
`seed2` arg is deprecated.Use sample_distorted_bounding_box_v2 instead.
W0408 19:36:34.523849 140553343391552 api.py:332] From /root/anaconda3/envs/tf1/lib/python3.6/site-packages/tensorflow_core/python/autograph/operators/control_flow.py:1004: sample_distorted_bounding_box (from tensorflow.python.ops.image_ops_impl) is deprecated and will be removed in a future version.
Instructions for updating:
`seed2` arg is deprecated.Use sample_distorted_bounding_box_v2 instead.
WARNING:tensorflow:From /root/anaconda3/envs/tf1/lib/python3.6/site-packages/object_detection/inputs.py:282: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
W0408 19:36:37.794182 140553343391552 deprecation.py:323] From /root/anaconda3/envs/tf1/lib/python3.6/site-packages/object_detection/inputs.py:282: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
INFO:tensorflow:Calling model_fn.
I0408 19:36:41.447258 140553343391552 estimator.py:1148] Calling model_fn.
WARNING:tensorflow:From /root/anaconda3/envs/tf1/lib/python3.6/site-packages/tf_slim/layers/layers.py:2802: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `layer.__call__` method instead.
W0408 19:36:41.740483 140553343391552 deprecation.py:323] From /root/anaconda3/envs/tf1/lib/python3.6/site-packages/tf_slim/layers/layers.py:2802: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `layer.__call__` method instead.
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 19:36:44.422615 140553343391552 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 19:36:44.448537 140553343391552 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 19:36:44.473237 140553343391552 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 19:36:44.497899 140553343391552 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 19:36:44.522875 140553343391552 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 19:36:44.547588 140553343391552 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0
WARNING:tensorflow:From /root/anaconda3/envs/tf1/lib/python3.6/site-packages/tensorflow_core/python/training/rmsprop.py:119: calling Ones.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
W0408 19:36:50.300761 140553343391552 deprecation.py:506] From /root/anaconda3/envs/tf1/lib/python3.6/site-packages/tensorflow_core/python/training/rmsprop.py:119: calling Ones.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
INFO:tensorflow:Done calling model_fn.
I0408 19:36:56.719294 140553343391552 estimator.py:1150] Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
I0408 19:36:56.720463 140553343391552 basic_session_run_hooks.py:541] Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
I0408 19:37:00.901003 140553343391552 monitored_session.py:240] Graph was finalized.
2021-04-08 19:37:00.901417: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 AVX512F FMA
2021-04-08 19:37:00.915375: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2300000000 Hz
2021-04-08 19:37:00.921384: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55aafa799d00 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2021-04-08 19:37:00.921444: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
2021-04-08 19:37:00.923131: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2021-04-08 19:37:00.976282: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: TITAN Xp major: 6 minor: 1 memoryClockRate(GHz): 1.582
pciBusID: 0000:1e:00.0
2021-04-08 19:37:00.976922: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2021-04-08 19:37:00.980881: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2021-04-08 19:37:00.984547: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2021-04-08 19:37:00.985348: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2021-04-08 19:37:00.988312: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2021-04-08 19:37:00.990079: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2021-04-08 19:37:01.001160: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2021-04-08 19:37:01.012950: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2021-04-08 19:37:01.013081: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2021-04-08 19:37:01.291709: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-04-08 19:37:01.291754: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165] 0
2021-04-08 19:37:01.291785: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0: N
2021-04-08 19:37:01.300849: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 11299 MB memory) -> physical GPU (device: 0, name: TITAN Xp, pci bus id: 0000:1e:00.0, compute capability: 6.1)
2021-04-08 19:37:01.303098: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55ab0e7a4670 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2021-04-08 19:37:01.303132: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): TITAN Xp, Compute Capability 6.1
INFO:tensorflow:Running local_init_op.
I0408 19:37:11.309897 140553343391552 session_manager.py:500] Running local_init_op.
INFO:tensorflow:Done running local_init_op.
I0408 19:37:12.092763 140553343391552 session_manager.py:502] Done running local_init_op.
INFO:tensorflow:Saving checkpoints for 0 into models/my_ssd_inception_v2/model.ckpt.
I0408 19:37:24.229167 140553343391552 basic_session_run_hooks.py:606] Saving checkpoints for 0 into models/my_ssd_inception_v2/model.ckpt.
2021-04-08 19:37:42.652158: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2021-04-08 19:37:48.131007: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
INFO:tensorflow:loss = 26.353672, step = 0
I0408 19:37:50.209866 140553343391552 basic_session_run_hooks.py:262] loss = 26.353672, step = 0
INFO:tensorflow:global_step/sec: 2.45856
I0408 19:38:30.884657 140553343391552 basic_session_run_hooks.py:692] global_step/sec: 2.45856
INFO:tensorflow:loss = 7.0979624, step = 100 (40.679 sec)
I0408 19:38:30.887990 140553343391552 basic_session_run_hooks.py:260] loss = 7.0979624, step = 100 (40.679 sec)
INFO:tensorflow:global_step/sec: 3.5244
I0408 19:38:59.256149 140553343391552 basic_session_run_hooks.py:692] global_step/sec: 3.5244
INFO:tensorflow:loss = 6.423036, step = 200 (28.385 sec)
I0408 19:38:59.272437 140553343391552 basic_session_run_hooks.py:260] loss = 6.423036, step = 200 (28.385 sec)
INFO:tensorflow:global_step/sec: 3.44889
I0408 19:39:28.251045 140553343391552 basic_session_run_hooks.py:692] global_step/sec: 3.44889
INFO:tensorflow:loss = 6.2270827, step = 300 (28.982 sec)
I0408 19:39:28.253960 140553343391552 basic_session_run_hooks.py:260] loss = 6.2270827, step = 300 (28.982 sec)
INFO:tensorflow:global_step/sec: 3.51676
I0408 19:39:56.686080 140553343391552 basic_session_run_hooks.py:692] global_step/sec: 3.51676
INFO:tensorflow:loss = 6.3535028, step = 400 (28.434 sec)
I0408 19:39:56.688244 140553343391552 basic_session_run_hooks.py:260] loss = 6.3535028, step = 400 (28.434 sec)
INFO:tensorflow:global_step/sec: 3.53058
I0408 19:40:25.010002 140553343391552 basic_session_run_hooks.py:692] global_step/sec: 3.53058
INFO:tensorflow:loss = 5.5039124, step = 500 (28.324 sec)
I0408 19:40:25.011913 140553343391552 basic_session_run_hooks.py:260] loss = 5.5039124, step = 500 (28.324 sec)
INFO:tensorflow:global_step/sec: 3.52694
I0408 19:40:53.363433 140553343391552 basic_session_run_hooks.py:692] global_step/sec: 3.52694
INFO:tensorflow:loss = 5.1926427, step = 600 (28.354 sec)
I0408 19:40:53.366256 140553343391552 basic_session_run_hooks.py:260] loss = 5.1926427, step = 600 (28.354 sec)
INFO:tensorflow:global_step/sec: 3.50592
I0408 19:41:21.886609 140553343391552 basic_session_run_hooks.py:692] global_step/sec: 3.50592
INFO:tensorflow:loss = 5.6967077, step = 700 (28.533 sec)
I0408 19:41:21.899112 140553343391552 basic_session_run_hooks.py:260] loss = 5.6967077, step = 700 (28.533 sec)
INFO:tensorflow:global_step/sec: 3.51716
I0408 19:41:50.318632 140553343391552 basic_session_run_hooks.py:692] global_step/sec: 3.51716
INFO:tensorflow:loss = 4.9808583, step = 800 (28.422 sec)
I0408 19:41:50.321131 140553343391552 basic_session_run_hooks.py:260] loss = 4.9808583, step = 800 (28.422 sec)
INFO:tensorflow:global_step/sec: 3.55013
I0408 19:42:18.486627 140553343391552 basic_session_run_hooks.py:692] global_step/sec: 3.55013
INFO:tensorflow:loss = 5.414756, step = 900 (28.168 sec)
I0408 19:42:18.489443 140553343391552 basic_session_run_hooks.py:260] loss = 5.414756, step = 900 (28.168 sec)
INFO:tensorflow:global_step/sec: 3.54571
I0408 19:42:46.689790 140553343391552 basic_session_run_hooks.py:692] global_step/sec: 3.54571
INFO:tensorflow:loss = 4.1702933, step = 1000 (28.203 sec)
I0408 19:42:46.692520 140553343391552 basic_session_run_hooks.py:260] loss = 4.1702933, step = 1000 (28.203 sec)
INFO:tensorflow:global_step/sec: 3.56015
I0408 19:43:14.778345 140553343391552 basic_session_run_hooks.py:692] global_step/sec: 3.56015
INFO:tensorflow:loss = 4.250403, step = 1100 (28.088 sec)
I0408 19:43:14.780816 140553343391552 basic_session_run_hooks.py:260] loss = 4.250403, step = 1100 (28.088 sec)
INFO:tensorflow:global_step/sec: 3.4538
I0408 19:43:43.732146 140553343391552 basic_session_run_hooks.py:692] global_step/sec: 3.4538
INFO:tensorflow:loss = 4.8844323, step = 1200 (28.964 sec)
I0408 19:43:43.744886 140553343391552 basic_session_run_hooks.py:260] loss = 4.8844323, step = 1200 (28.964 sec)
INFO:tensorflow:global_step/sec: 3.50202
I0408 19:44:12.286877 140553343391552 basic_session_run_hooks.py:692] global_step/sec: 3.50202
INFO:tensorflow:loss = 5.226234, step = 1300 (28.544 sec)
I0408 19:44:12.289032 140553343391552 basic_session_run_hooks.py:260] loss = 5.226234, step = 1300 (28.544 sec)
INFO:tensorflow:global_step/sec: 3.51738
I0408 19:44:40.717238 140553343391552 basic_session_run_hooks.py:692] global_step/sec: 3.51738
INFO:tensorflow:loss = 4.5052533, step = 1400 (28.431 sec)
I0408 19:44:40.719830 140553343391552 basic_session_run_hooks.py:260] loss = 4.5052533, step = 1400 (28.431 sec)
INFO:tensorflow:global_step/sec: 3.55431
I0408 19:45:08.852223 140553343391552 basic_session_run_hooks.py:692] global_step/sec: 3.55431
INFO:tensorflow:loss = 4.34357, step = 1500 (28.135 sec)
I0408 19:45:08.855122 140553343391552 basic_session_run_hooks.py:260] loss = 4.34357, step = 1500 (28.135 sec)
INFO:tensorflow:global_step/sec: 3.49746
I0408 19:45:37.444391 140553343391552 basic_session_run_hooks.py:692] global_step/sec: 3.49746
INFO:tensorflow:loss = 4.2605658, step = 1600 (28.592 sec)
I0408 19:45:37.447230 140553343391552 basic_session_run_hooks.py:260] loss = 4.2605658, step = 1600 (28.592 sec)
INFO:tensorflow:global_step/sec: 3.53497
I0408 19:46:05.733187 140553343391552 basic_session_run_hooks.py:692] global_step/sec: 3.53497
INFO:tensorflow:loss = 3.5676794, step = 1700 (28.295 sec)
I0408 19:46:05.741827 140553343391552 basic_session_run_hooks.py:260] loss = 3.5676794, step = 1700 (28.295 sec)
INFO:tensorflow:global_step/sec: 3.55267
I0408 19:46:33.881087 140553343391552 basic_session_run_hooks.py:692] global_step/sec: 3.55267
INFO:tensorflow:loss = 4.289984, step = 1800 (28.142 sec)
I0408 19:46:33.884087 140553343391552 basic_session_run_hooks.py:260] loss = 4.289984, step = 1800 (28.142 sec)
INFO:tensorflow:global_step/sec: 3.53733
I0408 19:47:02.150989 140553343391552 basic_session_run_hooks.py:692] global_step/sec: 3.53733
INFO:tensorflow:loss = 3.744223, step = 1900 (28.270 sec)
I0408 19:47:02.153777 140553343391552 basic_session_run_hooks.py:260] loss = 3.744223, step = 1900 (28.270 sec)
INFO:tensorflow:Saving checkpoints for 1997 into models/my_ssd_inception_v2/model.ckpt.
I0408 19:47:29.373254 140553343391552 basic_session_run_hooks.py:606] Saving checkpoints for 1997 into models/my_ssd_inception_v2/model.ckpt.
INFO:tensorflow:Reading unweighted datasets: ['annotations/test.record']
I0408 19:47:43.834930 140553343391552 dataset_builder.py:163] Reading unweighted datasets: ['annotations/test.record']
INFO:tensorflow:Reading record datasets for input file: ['annotations/test.record']
I0408 19:47:43.839394 140553343391552 dataset_builder.py:80] Reading record datasets for input file: ['annotations/test.record']
INFO:tensorflow:Number of filenames to read: 1
I0408 19:47:43.839524 140553343391552 dataset_builder.py:81] Number of filenames to read: 1
INFO:tensorflow:Calling model_fn.
I0408 19:47:45.471107 140553343391552 estimator.py:1148] Calling model_fn.
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 19:47:47.676634 140553343391552 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 19:47:47.701958 140553343391552 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 19:47:47.727361 140553343391552 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 19:47:47.752024 140553343391552 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 19:47:47.777025 140553343391552 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 19:47:47.801182 140553343391552 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0
WARNING:tensorflow:From /root/anaconda3/envs/tf1/lib/python3.6/site-packages/object_detection/eval_util.py:929: to_int64 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
W0408 19:47:48.630852 140553343391552 deprecation.py:323] From /root/anaconda3/envs/tf1/lib/python3.6/site-packages/object_detection/eval_util.py:929: to_int64 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
WARNING:tensorflow:From /root/anaconda3/envs/tf1/lib/python3.6/site-packages/object_detection/utils/visualization_utils.py:618: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.
Instructions for updating:
tf.py_func is deprecated in TF V2. Instead, there are twooptions available in V2.- tf.py_function takes a python function which manipulates tf eagertensors instead of numpy arrays. It's easy to convert a tf eager tensor toan ndarray (just call tensor.numpy()) but having access to eager tensorsmeans `tf.py_function`s can use accelerators such as GPUs as well asbeing differentiable using a gradient tape.- tf.numpy_function maintains the semantics of the deprecated tf.py_func(it is not differentiable, and manipulates numpy arrays). It drops thestateful argument making all functions stateful.W0408 19:47:48.957098 140553343391552 deprecation.py:323] From /root/anaconda3/envs/tf1/lib/python3.6/site-packages/object_detection/utils/visualization_utils.py:618: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.
Instructions for updating:
tf.py_func is deprecated in TF V2. Instead, there are twooptions available in V2.- tf.py_function takes a python function which manipulates tf eagertensors instead of numpy arrays. It's easy to convert a tf eager tensor toan ndarray (just call tensor.numpy()) but having access to eager tensorsmeans `tf.py_function`s can use accelerators such as GPUs as well asbeing differentiable using a gradient tape.- tf.numpy_function maintains the semantics of the deprecated tf.py_func(it is not differentiable, and manipulates numpy arrays). It drops thestateful argument making all functions stateful.INFO:tensorflow:Done calling model_fn.
I0408 19:47:49.616228 140553343391552 estimator.py:1150] Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2021-04-08T19:47:49Z
I0408 19:47:49.635172 140553343391552 evaluation.py:255] Starting evaluation at 2021-04-08T19:47:49Z
INFO:tensorflow:Graph was finalized.
I0408 19:47:50.092404 140553343391552 monitored_session.py:240] Graph was finalized.
2021-04-08 19:47:50.101238: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: TITAN Xp major: 6 minor: 1 memoryClockRate(GHz): 1.582
pciBusID: 0000:1e:00.0
2021-04-08 19:47:50.101469: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2021-04-08 19:47:50.101504: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2021-04-08 19:47:50.101532: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2021-04-08 19:47:50.101561: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2021-04-08 19:47:50.101586: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2021-04-08 19:47:50.101615: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2021-04-08 19:47:50.101686: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2021-04-08 19:47:50.111780: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2021-04-08 19:47:50.111874: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-04-08 19:47:50.111910: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165] 0
2021-04-08 19:47:50.111944: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0: N
2021-04-08 19:47:50.116084: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 11299 MB memory) -> physical GPU (device: 0, name: TITAN Xp, pci bus id: 0000:1e:00.0, compute capability: 6.1)
INFO:tensorflow:Restoring parameters from models/my_ssd_inception_v2/model.ckpt-1997
I0408 19:47:50.128154 140553343391552 saver.py:1284] Restoring parameters from models/my_ssd_inception_v2/model.ckpt-1997
INFO:tensorflow:Running local_init_op.
I0408 19:47:52.254904 140553343391552 session_manager.py:500] Running local_init_op.
INFO:tensorflow:Done running local_init_op.
I0408 19:47:52.436636 140553343391552 session_manager.py:502] Done running local_init_op.
INFO:tensorflow:Performing evaluation on 25 images.
I0408 19:47:56.768712 140496349411072 coco_evaluation.py:293] Performing evaluation on 25 images.
creating index...
index created!
INFO:tensorflow:Loading and preparing annotation results...
I0408 19:47:56.769833 140496349411072 coco_tools.py:116] Loading and preparing annotation results...
INFO:tensorflow:DONE (t=0.00s)
I0408 19:47:56.772033 140496349411072 coco_tools.py:138] DONE (t=0.00s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=0.22s).
Accumulating evaluation results...
DONE (t=0.03s).Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.001Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.007Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.000Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.000Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.002Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.021Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.022Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.021
INFO:tensorflow:Finished evaluation at 2021-04-08-19:47:57
I0408 19:47:57.225715 140553343391552 evaluation.py:275] Finished evaluation at 2021-04-08-19:47:57
INFO:tensorflow:Saving dict for global step 1997: DetectionBoxes_Precision/mAP = 0.0001709473, DetectionBoxes_Precision/mAP (large) = 0.00013584658, DetectionBoxes_Precision/mAP (medium) = 0.0073927394, DetectionBoxes_Precision/mAP (small) = -1.0, DetectionBoxes_Precision/mAP@.50IOU = 0.00088169175, DetectionBoxes_Precision/mAP@.75IOU = 0.0, DetectionBoxes_Recall/AR@1 = 0.0, DetectionBoxes_Recall/AR@10 = 0.002352941, DetectionBoxes_Recall/AR@100 = 0.02117647, DetectionBoxes_Recall/AR@100 (large) = 0.020967742, DetectionBoxes_Recall/AR@100 (medium) = 0.02173913, DetectionBoxes_Recall/AR@100 (small) = -1.0, Loss/classification_loss = 8.233138, Loss/localization_loss = 4.0231056, Loss/regularization_loss = 0.48583883, Loss/total_loss = 12.742084, global_step = 1997, learning_rate = 0.004, loss = 12.742084
I0408 19:47:57.226226 140553343391552 estimator.py:2049] Saving dict for global step 1997: DetectionBoxes_Precision/mAP = 0.0001709473, DetectionBoxes_Precision/mAP (large) = 0.00013584658, DetectionBoxes_Precision/mAP (medium) = 0.0073927394, DetectionBoxes_Precision/mAP (small) = -1.0, DetectionBoxes_Precision/mAP@.50IOU = 0.00088169175, DetectionBoxes_Precision/mAP@.75IOU = 0.0, DetectionBoxes_Recall/AR@1 = 0.0, DetectionBoxes_Recall/AR@10 = 0.002352941, DetectionBoxes_Recall/AR@100 = 0.02117647, DetectionBoxes_Recall/AR@100 (large) = 0.020967742, DetectionBoxes_Recall/AR@100 (medium) = 0.02173913, DetectionBoxes_Recall/AR@100 (small) = -1.0, Loss/classification_loss = 8.233138, Loss/localization_loss = 4.0231056, Loss/regularization_loss = 0.48583883, Loss/total_loss = 12.742084, global_step = 1997, learning_rate = 0.004, loss = 12.742084
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 1997: models/my_ssd_inception_v2/model.ckpt-1997
I0408 19:47:58.198050 140553343391552 estimator.py:2109] Saving 'checkpoint_path' summary for global step 1997: models/my_ssd_inception_v2/model.ckpt-1997
INFO:tensorflow:Saving checkpoints for 2000 into models/my_ssd_inception_v2/model.ckpt.
I0408 19:47:59.095469 140553343391552 basic_session_run_hooks.py:606] Saving checkpoints for 2000 into models/my_ssd_inception_v2/model.ckpt.
INFO:tensorflow:Skip the current checkpoint eval due to throttle secs (600 secs).
I0408 19:48:01.902159 140553343391552 training.py:527] Skip the current checkpoint eval due to throttle secs (600 secs).
INFO:tensorflow:Reading unweighted datasets: ['annotations/test.record']
I0408 19:48:01.939406 140553343391552 dataset_builder.py:163] Reading unweighted datasets: ['annotations/test.record']
INFO:tensorflow:Reading record datasets for input file: ['annotations/test.record']
I0408 19:48:01.946810 140553343391552 dataset_builder.py:80] Reading record datasets for input file: ['annotations/test.record']
INFO:tensorflow:Number of filenames to read: 1
I0408 19:48:01.946915 140553343391552 dataset_builder.py:81] Number of filenames to read: 1
INFO:tensorflow:Calling model_fn.
I0408 19:48:02.928972 140553343391552 estimator.py:1148] Calling model_fn.
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 19:48:05.048127 140553343391552 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 19:48:05.074190 140553343391552 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 19:48:05.100624 140553343391552 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 19:48:05.125298 140553343391552 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 19:48:05.150101 140553343391552 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 19:48:05.175326 140553343391552 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0
INFO:tensorflow:Done calling model_fn.
I0408 19:48:06.351978 140553343391552 estimator.py:1150] Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2021-04-08T19:48:06Z
I0408 19:48:06.365622 140553343391552 evaluation.py:255] Starting evaluation at 2021-04-08T19:48:06Z
INFO:tensorflow:Graph was finalized.
I0408 19:48:06.785058 140553343391552 monitored_session.py:240] Graph was finalized.
2021-04-08 19:48:06.787840: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: TITAN Xp major: 6 minor: 1 memoryClockRate(GHz): 1.582
pciBusID: 0000:1e:00.0
2021-04-08 19:48:06.788024: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2021-04-08 19:48:06.788048: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2021-04-08 19:48:06.788072: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2021-04-08 19:48:06.788094: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2021-04-08 19:48:06.788114: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2021-04-08 19:48:06.788137: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2021-04-08 19:48:06.788157: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2021-04-08 19:48:06.791190: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2021-04-08 19:48:06.791261: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-04-08 19:48:06.791276: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165] 0
2021-04-08 19:48:06.791332: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0: N
2021-04-08 19:48:06.794789: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 11299 MB memory) -> physical GPU (device: 0, name: TITAN Xp, pci bus id: 0000:1e:00.0, compute capability: 6.1)
INFO:tensorflow:Restoring parameters from models/my_ssd_inception_v2/model.ckpt-2000
I0408 19:48:06.804881 140553343391552 saver.py:1284] Restoring parameters from models/my_ssd_inception_v2/model.ckpt-2000
INFO:tensorflow:Running local_init_op.
I0408 19:48:08.983566 140553343391552 session_manager.py:500] Running local_init_op.
INFO:tensorflow:Done running local_init_op.
I0408 19:48:09.121775 140553343391552 session_manager.py:502] Done running local_init_op.
INFO:tensorflow:Performing evaluation on 25 images.
I0408 19:48:13.511092 140495493789440 coco_evaluation.py:293] Performing evaluation on 25 images.
creating index...
index created!
INFO:tensorflow:Loading and preparing annotation results...
I0408 19:48:13.512457 140495493789440 coco_tools.py:116] Loading and preparing annotation results...
INFO:tensorflow:DONE (t=0.00s)
I0408 19:48:13.514708 140495493789440 coco_tools.py:138] DONE (t=0.00s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=0.25s).
Accumulating evaluation results...
DONE (t=0.01s).Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.001Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.009Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.000Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.000Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.000Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.027Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.017Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.031
INFO:tensorflow:Finished evaluation at 2021-04-08-19:48:13
I0408 19:48:13.930575 140553343391552 evaluation.py:275] Finished evaluation at 2021-04-08-19:48:13
INFO:tensorflow:Saving dict for global step 2000: DetectionBoxes_Precision/mAP = 0.000258673, DetectionBoxes_Precision/mAP (large) = 0.00023983713, DetectionBoxes_Precision/mAP (medium) = 0.009405941, DetectionBoxes_Precision/mAP (small) = -1.0, DetectionBoxes_Precision/mAP@.50IOU = 0.001324426, DetectionBoxes_Precision/mAP@.75IOU = 0.0, DetectionBoxes_Recall/AR@1 = 0.0, DetectionBoxes_Recall/AR@10 = 0.0, DetectionBoxes_Recall/AR@100 = 0.027058823, DetectionBoxes_Recall/AR@100 (large) = 0.030645162, DetectionBoxes_Recall/AR@100 (medium) = 0.017391304, DetectionBoxes_Recall/AR@100 (small) = -1.0, Loss/classification_loss = 7.987997, Loss/localization_loss = 4.0830684, Loss/regularization_loss = 0.48584178, Loss/total_loss = 12.556907, global_step = 2000, learning_rate = 0.004, loss = 12.556907
I0408 19:48:13.931072 140553343391552 estimator.py:2049] Saving dict for global step 2000: DetectionBoxes_Precision/mAP = 0.000258673, DetectionBoxes_Precision/mAP (large) = 0.00023983713, DetectionBoxes_Precision/mAP (medium) = 0.009405941, DetectionBoxes_Precision/mAP (small) = -1.0, DetectionBoxes_Precision/mAP@.50IOU = 0.001324426, DetectionBoxes_Precision/mAP@.75IOU = 0.0, DetectionBoxes_Recall/AR@1 = 0.0, DetectionBoxes_Recall/AR@10 = 0.0, DetectionBoxes_Recall/AR@100 = 0.027058823, DetectionBoxes_Recall/AR@100 (large) = 0.030645162, DetectionBoxes_Recall/AR@100 (medium) = 0.017391304, DetectionBoxes_Recall/AR@100 (small) = -1.0, Loss/classification_loss = 7.987997, Loss/localization_loss = 4.0830684, Loss/regularization_loss = 0.48584178, Loss/total_loss = 12.556907, global_step = 2000, learning_rate = 0.004, loss = 12.556907
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 2000: models/my_ssd_inception_v2/model.ckpt-2000
I0408 19:48:13.956167 140553343391552 estimator.py:2109] Saving 'checkpoint_path' summary for global step 2000: models/my_ssd_inception_v2/model.ckpt-2000
INFO:tensorflow:Performing the final export in the end of training.
I0408 19:48:13.959100 140553343391552 exporter.py:410] Performing the final export in the end of training.
INFO:tensorflow:Calling model_fn.
I0408 19:48:14.372867 140553343391552 estimator.py:1148] Calling model_fn.
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 19:48:16.682502 140553343391552 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 19:48:16.708631 140553343391552 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 19:48:16.733302 140553343391552 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 19:48:16.758555 140553343391552 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 19:48:16.782882 140553343391552 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0
INFO:tensorflow:depth of additional conv before box predictor: 0
I0408 19:48:16.807321 140553343391552 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0
INFO:tensorflow:Done calling model_fn.
I0408 19:48:17.382560 140553343391552 estimator.py:1150] Done calling model_fn.
WARNING:tensorflow:From /root/anaconda3/envs/tf1/lib/python3.6/site-packages/tensorflow_core/python/saved_model/signature_def_utils_impl.py:201: build_tensor_info (from tensorflow.python.saved_model.utils_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.utils.build_tensor_info or tf.compat.v1.saved_model.build_tensor_info.
W0408 19:48:17.382837 140553343391552 deprecation.py:323] From /root/anaconda3/envs/tf1/lib/python3.6/site-packages/tensorflow_core/python/saved_model/signature_def_utils_impl.py:201: build_tensor_info (from tensorflow.python.saved_model.utils_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.utils.build_tensor_info or tf.compat.v1.saved_model.build_tensor_info.
INFO:tensorflow:Signatures INCLUDED in export for Classify: None
I0408 19:48:17.383411 140553343391552 export_utils.py:170] Signatures INCLUDED in export for Classify: None
INFO:tensorflow:Signatures INCLUDED in export for Regress: None
I0408 19:48:17.383496 140553343391552 export_utils.py:170] Signatures INCLUDED in export for Regress: None
INFO:tensorflow:Signatures INCLUDED in export for Predict: ['tensorflow/serving/predict', 'serving_default']
I0408 19:48:17.383557 140553343391552 export_utils.py:170] Signatures INCLUDED in export for Predict: ['tensorflow/serving/predict', 'serving_default']
INFO:tensorflow:Signatures INCLUDED in export for Train: None
I0408 19:48:17.383643 140553343391552 export_utils.py:170] Signatures INCLUDED in export for Train: None
INFO:tensorflow:Signatures INCLUDED in export for Eval: None
I0408 19:48:17.383695 140553343391552 export_utils.py:170] Signatures INCLUDED in export for Eval: None
2021-04-08 19:48:17.385852: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: TITAN Xp major: 6 minor: 1 memoryClockRate(GHz): 1.582
pciBusID: 0000:1e:00.0
2021-04-08 19:48:17.385957: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2021-04-08 19:48:17.385978: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2021-04-08 19:48:17.385999: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2021-04-08 19:48:17.386017: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2021-04-08 19:48:17.386034: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2021-04-08 19:48:17.386053: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2021-04-08 19:48:17.386070: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2021-04-08 19:48:17.389097: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2021-04-08 19:48:17.389145: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-04-08 19:48:17.389159: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165] 0
2021-04-08 19:48:17.389171: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0: N
2021-04-08 19:48:17.392436: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 11299 MB memory) -> physical GPU (device: 0, name: TITAN Xp, pci bus id: 0000:1e:00.0, compute capability: 6.1)
INFO:tensorflow:Restoring parameters from models/my_ssd_inception_v2/model.ckpt-2000
I0408 19:48:17.407397 140553343391552 saver.py:1284] Restoring parameters from models/my_ssd_inception_v2/model.ckpt-2000
INFO:tensorflow:Assets added to graph.
I0408 19:48:18.091548 140553343391552 builder_impl.py:665] Assets added to graph.
INFO:tensorflow:No assets to write.
I0408 19:48:18.091738 140553343391552 builder_impl.py:460] No assets to write.
INFO:tensorflow:SavedModel written to: models/my_ssd_inception_v2/export/Servo/temp-b'1617882493'/saved_model.pb
I0408 19:48:20.169840 140553343391552 builder_impl.py:425] SavedModel written to: models/my_ssd_inception_v2/export/Servo/temp-b'1617882493'/saved_model.pb
INFO:tensorflow:Loss for final step: 3.5268488.
I0408 19:48:21.147778 140553343391552 estimator.py:371] Loss for final step: 3.5268488.跑完后的结果使用tensorfboard进行可视化，结果如下：
准确率曲线：
召回率曲线：
损失曲线：
迭代20W次后的验证集结果：
左右两张图是一致的，左图是模型测试结果，右图是真值（人为标的），进行对比可得准确率。模型准确率较低，主要原因是训练的数据太小，只有三四百张图片，而且标注的目标特征较复杂，学习起来难度较大，导致模型准确率低。从训练的准确率也可以看出，数据量小，迭代次数太多没有意义，反而会过拟合，导致准确率下降。
Ubuntu16.04+cuda10.0+cudnn7.6配置tensorflow1.x环境相关推荐
- Ubuntu16.04 + Cuda-9.0 + Cudnn-7.1.4 + TensorFlow1.8(极其简单)
步骤 1.Ubuntu16.04 LTS 2.配置Nvidia显卡驱动 3.Cuda-9.0 4.Cudnn-7.1.4 5.TensorFlow1.8 1. Ubuntu16.04 LTS 安装Ub ...
- docker 配置使用宿主机的GPU（ubuntu16.04+cuda10.0+cudnn7）
1. 安装 Docker 卸载旧版本 Docker sudo apt-get remove docker docker-engine docker.io containerd runc 安装新版本 s ...
- 成功安装ubuntu16.04+cuda9.0+cuDNN7.0.5+caffe+digits（附错误汇总）
成功安装ubuntu16.04+cuda9.0+cuDNN7.0.5+caffe+digits(附错误汇总) 本机硬件:双核i7 内存8G 显卡GTX070(不好意思,比较水) 安装顺序:依赖包--& ...
- NVIDA GTX2080 Ubuntu18.04 cuda10.0 cudnn7.5 tensorflow 1.12 install
NVIDA GTX2080 Ubuntu18.04 cuda10.0 cudnn7.5 tensorflow 1.12 install 心仪已久的显卡2080终于到了,Ubuntu18这么帅,怎么还能 ...
- DL之IDE：深度学习之计算机视觉开发环境搭建的详细流程(Ubuntu16.04+cuda9.0+cuDNN7.4.2+tensorflow_gpu)
DL之IDE:深度学习之计算机视觉开发环境搭建的详细流程(Ubuntu16.04+cuda9.0+cuDNN7.4.2+tensorflow_gpu) 目录 1.安装nvidia驱动 2.安装CUDA ...
- 深度学习环境配置 (Ubuntu18.04 + CUDA10.0 + cuDNN7.6.5 + TensorFlow2.0)
@ Bergen, Norway 第一次安装 CUDA 的过程简直抓狂,中间出现了很多次莫名其妙的 bug,踩了很多坑.比如装好了 CUDA 重启后进不去桌面系统了,直接黑屏.比如鼠标键盘都不 wor ...
- ubuntu16.04+cuda9.0+cudnn7.0+caffe
准备工作 删除原有cuda sudo apt autoremove cuda 删除原有软件更新 系统设置-软件和更新-其他软件 cuda9.0 下载链接: cuda9.0 在下载目录打开termina ...
- cuda8.0.44linux.run,ubuntu16.04 cuda8.0 cudnn7.0 安装
cuda 8.0 安装: sudo sh ./cuda_8.0.44_linux.run (可通过sudo sh ./cuda_8.0.44_linux.run --help查看一些参数命令,安装某一 ...
- ubuntu16.04+cuda8.0+cudnn5.1+caffe 安装（基于独立主机集成显卡GTX1080Ti）
最近开始配置自己的深度学习主机,由于之前只在笔记本上和单显卡主机上安装过,所以可以说是zero经验,以至于很多问题不了解都当成了bug,故为了方便以后的自己以及大众,特在此写下post一篇.(由于可能 ...
最新文章
- jQuery快速入门
- node.js博客GitHub搭建（hexo）
- LoadRunner监控局域网内其他服务器系统资源设置
- 代码的简单就在于——直接能看懂
- html6与html5的区别,XHTML和HTML5的区别
- 20165236 2017-2018-2 《Java程序设计》结对编程练习_四则运算
- 高等学校计算机房使用章程,普通高等学校设置暂行条例.DOC
- Flutter的Chip标签组件
- VisualVM安装,插件安装,各个面板信息讲解
- python保存到txt_python保存成文本文件
- 迪文串口屏TTL与主控板RS232电平信号转换方案
- 【Windows】文件夹被占用删除不掉
- 创建Angular项目及常用命令
- 爬取今日头条街拍美女图片
- php出现网页无法运作怎么处理,该网页无法正常运作怎么解决？
- 相忘江湖不如相濡以沫(Ⅰ)
- 蓝牙协议栈消息的关联
- python将文字转换为语音_python把文字转成语音
- 最重要的技术深入学习
- 全站写作平台疯传24小时删除
热门文章
- jeecg-mybatis-framework 参考学习版本
- 数据源管理 | 分布式NoSQL系统，Cassandra集群管理
- form表单序列化成json数据 将空值用空字符串代替（form表单中checkBox数据会用逗号隔开拼接成字符串）...
- 上传一个 游戏server架构图
- [Selenium] 如何在老版本的Chrome 浏览器上使用selenium
- DedeCMS四类核心表
- css碎步测量,CORS系统控制点点位可靠性研究.doc
- linux 关闭句柄,主库出现大量的未关闭句柄，对应linux命令 ： lsof -p 28314|grep CLOSE_WAIT...
- batchplot插件用法_Batchplot怎么安装及使用？Batchplot的安装方法及使用方法介绍
- css 透明叠加_细品CSS（二）