Skip to main content
 首页 » 编程设计

python之tensorflow 运行永远需要小数据

2024年08月21日30虾米哥

我正在尝试一本书的一个简单示例,其中我有一个包含 892 行的火车数据样本,这是常见的泰坦尼克号生存教科书示例。我定义:

def read_csv(batch_size, file_path, record_defaults): 
 
    filename_queue = tf.train.string_input_producer([file_path]) 
    reader = tf.TextLineReader(skip_header_lines=1) 
    key, value = reader.read(filename_queue) 
 
    # decode_csv will convert a Tensor from type string (the text line) in 
    # a tuple of tensor columns with the specified defaults, which also 
    # sets the data type for each column 
    decoded = tf.decode_csv(value, record_defaults=record_defaults) 
 
 
    # batch actually reads the file and loads "batch_size" rows in a single tensor 
    return tf.train.shuffle_batch(decoded, 
                                  batch_size=batch_size, 
                                  capacity=batch_size * 50, 
                                  min_after_dequeue=batch_size) 
def inputs(): 
    passenger_id, survived, pclass, name, sex, age, sibsp, parch, ticket, fare, cabin, embarked = \ 
        read_csv(BATCH_SIZE, file_path, record_defaults) 
 
    # convert categorical data 
    is_first_class = tf.to_float(tf.equal(pclass, [1])) 
    is_second_class = tf.to_float(tf.equal(pclass, [2])) 
    is_third_class = tf.to_float(tf.equal(pclass, [3])) 
 
    gender = tf.to_float(tf.equal(sex, ["female"])) 
 
    # Finally we pack all the features in a single matrix; 
    # We then transpose to have a matrix with one example per row and one feature per column. 
    features = tf.transpose(tf.pack([is_first_class, is_second_class, is_third_class, gender, age])) 
 
 
    print 'shape of features', features.get_shape() 
    return features, survived 

现在我尝试做:

graph = tf.Graph() 
 
with tf.Session(graph=graph) as sess: 
 
    W = tf.Variable(tf.zeros([5, 1]), name="weights") 
    b = tf.Variable(0., name="bias") 
    tf.global_variables_initializer().run() 
    print 'tf was run' 
    X,Y = inputs() 
    print 'inputs!' 
    sess.run(Y) 

我明白了

'tf was run!' 
'inputs!' 

但是 run 部分永远挂起(或者至少挂起很长时间)。我正在使用 2.7 内核和 tf 版本 0.12

在 Jupyter 上运行它

我错过了什么?

请您参考如下方法:

在线

return tf.train.shuffle_batch(decoded, 
                              batch_size=batch_size, 
                              capacity=batch_size * 50, 
                              min_after_dequeue=batch_size) 

您要定义从队列中提取值和创建批处理的操作。

如果您查看该方法的完整签名,您会注意到有一个参数引用了多个线程。

tf.train.shuffle_batch( 
tensors, 
batch_size, 
capacity, 
min_after_dequeue, 
num_threads=1, 
seed=None, enqueue_many=False, shapes=None, 
allow_smaller_final_batch=False, shared_name=None, name=None) 

我指出这一点是因为您定义的操作是从某些线程执行的。 线程必须 启动和停止,此函数不会为您执行此操作。关于线程处理,此函数所做的唯一事情是将 num_thread 添加到队列中。

实际上,要启动和停止线程,您需要在 session 中定义一个唤醒队列中线程的操作:

graph = tf.Graph() 
 
with tf.Session(graph=graph) as sess: 
 
    W = tf.Variable(tf.zeros([5, 1]), name="weights") 
    b = tf.Variable(0., name="bias") 
    tf.global_variables_initializer().run() 
    print 'tf was run' 
    X,Y = inputs() 
    # define a coordinator to start and stop the threads 
    coord = tf.train.Coordinator() 
    # wake up the threads 
    threads = tf.train.start_queue_runners(sess=sess, coord=coord) 
    print 'inputs!' 
    sess.run(Y) #execute operation 
    # When done, ask the threads to stop. 
    coord.request_stop() 
    # Wait for threads to finish. 
    coord.join(threads)