tensorflow quene queueRunner

转自：http://blog.csdn.net/lujiandong1/article/details/53369961

队列本身也是图中的一个节点。其他节点（enqueue, dequeue）可以修改队列节点中的内容。

#-*- coding:utf-8 -*-
import tensorflow as tf  #创建的图:一个先入先出队列,以及初始化,出队,+1,入队操作
q = tf.FIFOQueue(3, "float")
init = q.enqueue_many(([0.1, 0.2, 0.3],))  #一次往队列中输入多个值
x = q.dequeue()  #从队列中取出一个值
y = x + 1
q_inc = q.enqueue([y])  #往队列中push一个值#开启一个session,session是会话,会话的潜在含义是状态保持,各种tensor的状态保持
with tf.Session() as sess:  sess.run(init)  for i in range(2):  sess.run(q_inc)  quelen =  sess.run(q.size())  for i in range(quelen):  print (sess.run(q.dequeue()))

https://indico.io/blog/tensorflow-data-inputs-part1-placeholders-protobufs-queues/
队列

A TensorFlow queue is quite similar to a regular queue, they are
symbolic and only performed on Tensorflow’s graph. As part of the API,
a TFRecordReader always acts on a queue of filenames. It will pop a
filename off the queue and use that filename until the tfrecord is
empty. At this point it will grab the next filename off the filename
queue. Both the queue and the TFRecordReader have some state to keep
track of where they are. The only piece missing is how the filename
queue is actually fed. On initialization it is empty. This is where
the concept of QueueRunners comes in. There’s nothing magical about
it. At its core, it is simply a thread that uses a session and calls
an enqueue op over and over again.

TFRecordReader是在一个文件队列上操作。从队列中取出一个文件名，读取完这个文件里内容后又取出另一个文件名，直接队列为空。队列最开始是空的，队列的填充是通过QueueRunners来实现的。QueueRunners是一个线程，他会重复调用入队列操作来填充队列。

We do, however, have to signal to TensorFlow to start these threads.
Without this, the queue will be blocked indefinitely, waiting for data
to be enqueued. To start the QueueRunners you can call
tf.train.start_queue_runners(sess=sess). This call is not symbolic. It
goes and creates threads. It is important to note that this must be
called after the initialization op is run. These threads try to
enqueue data to queues. If the queues are not initialized, TensorFlow
will rightly throw an error.

在代码里需要显示的启动tensorflow里的线程，否则队列一直为空，处于阻塞状态。可以在初始化操作函数执行后调用tf.train.start_queue_runners(sess=sess)来启动线程，这个函数会启动图里面的QueueRunners集合。

batch的实现

Great! Now we have two Tensors that represents a single example. We
could train with this using gradient descent, but that is rather
silly. It has been shown that training with batches of examples work
far better than training with a single examples.

# get single examples
label, image = read_and_decode_single_example("mnist.tfrecords")
# 返回的images_batch是一个list，代表一个batch，batch_size=128
images_batch, labels_batch = tf.train.shuffle_batch([image, label], batch_size=128,capacity=2000,min_after_dequeue=1000)sess = tf.Session()
init = tf.initialize_all_variables()
sess.run(init)
tf.train.start_queue_runners(sess=sess)#启动图中的QueueRunners集合
labels, images= sess.run([labels_batch, images_batch])

As discussed above, if label and image represent a single example, how can we make a batch of different examples? The answer lies in another set of queues and QueueRunners. shuffle_batch constructs a RandomShuffleQueue and proceeds to fill it with individual image and labels. This filling is done on a separate thread with a QueueRunner. The RandomShuffleQueue accumulates examples sequentially until it contains batch_size +min_after_dequeue examples are present. It then selects batch_size random elements from the queue to return. The value actually returned by shuffle_batch is the result of a dequeue_many call on the RandomShuffleQueue.

tensorflow quene queueRunner相关推荐

tensorflow随笔 -QueueRunner
使用基于队列的api实现输入管道,属于数据API. 从文件中读取记录的典型基于队列的管道有以下几个阶段: 1.文件名列表 2.可选的文件名洗牌式重排 3.可选的epoch限制 4.文件名队列 5.文件 ...
tensorflow随笔-队列管理器QueueRunner－生产者与消费者
# -*- coding: utf-8 -*- """ Spyder EditorThis is a temporary script file. "" ...
Tensorflow多线程输入数据处理框架（一）——队列与多线程
参考书 <TensorFlow:实战Google深度学习框架>(第2版) 对于队列,修改队列状态的操作主要有Enqueue.EnqueueMany和Dequeue.以下程序展示了如何使用这 ...
TensorFlow高效读取数据的方法
概述最新上传的mcnn中有完整的数据读写示例,可以参考. 关于Tensorflow读取数据,官网给出了三种方法: 供给数据(Feeding): 在TensorFlow程序运行的每一步, 让Pytho ...
由浅入深之Tensorflow(3)----数据读取之TFRecords
由浅入深之Tensorflow(3)----数据读取之TFRecords 转载自http://blog.csdn.net/u012759136/article/details/52232266 原文作 ...
『TensorFlow』第十一弹_队列多线程TFRecod文件_我辈当高歌
TF数据读取队列机制详解一.TFR文件多线程队列读写操作 TFRecod文件写入操作 import tensorflow as tf def _int64_feature(value):# valu ...
7.3 TensorFlow笔记(基础篇):加载数据之从队列中读取
前言整体步骤在TensorFlow中进行模型训练时,在官网给出的三种读取方式,中最好的文件读取方式就是将利用队列进行文件读取,而且步骤有两步: 1. 把样本数据写入TFRecords二进制文件 2 ...
6.1 Tensorflow笔记(基础篇):队列与线程
前言在Tensorflow的实际应用中,队列与线程是必不可少,主要应用于数据的加载等,不同的情况下使用不同的队列,主线程与其他线程异步进行数据的训练与读取,所以队列与线程的知识也是Tensorflo ...
【Tensorflow】io 操作
文章首发于微信公众号<有三AI> [从caffe到Tensorflow 1]io 操作最近项目要频繁用到tensorflow,所以不得不认真研究下tensorflow而不是跟之前一样遇到 ...

tensorflow quene queueRunner

tensorflow quene queueRunner相关推荐

最新文章

热门文章