Introduction to Tensorflow
Introduction to Tensorflow
Basic Usage
To use TensorFlow you need to understand how TensorFlow:
Graphs
TensorFlow is a programming system in which you represent computations as graphs.
-
Construction phase.
- <u>Assemble a graph</u>. (<u>Create a graph</u> to represent and <u>train a neural network</u>.)
-
Execution phase.
- <u>Use a session to execute ops (Nodes in the graph)</u>. (Repeatedly <u>execute</u> a set of training <u>ops</u>.)
- An op takes zero or more
Tensors
, performs some computation, and produces zero or moreTensors
.
Sessions
Executes graphs in the context of Sessions.
- To compute anything, a graph must be launched in a
Session
. - A
Session
places the graph ops ontoDevices
, such as CPUs or GPUs, and provides methods to execute them. These methods return tensors produced by ops as numpy (the fundamental package for scientific computing with Python)ndarray
objects in Python.
Tensors
Represents data as tensors.
- TensorFlow programs use a tensor data structure to represent all data.
- You can think of a TensorFlow tensor as an n-dimensional array or list.
- For example, you can represent a mini-batch of images as a 4-D array of floating point numbers with dimensions
[batch, height, width, channels]
.
- For example, you can represent a mini-batch of images as a 4-D array of floating point numbers with dimensions
- A tensor has a <u>static type</u> and <u>dynamic dimensions</u>. Only tensors may be passed between nodes in the computation graph.
Rank
- Tensors are described by a unit of dimensionality known as rank.
- Tensor rank is <u>not</u> the same as matrix rank.
- Tensor rank (sometimes referred to as order or degree or n-dimension) is the number of dimensions of the tensor.
Rank | Math entity | Python example |
---|---|---|
0 | Scalar (magnitude only) | s = 483 |
1 | Vector (magnitude and direction) | v = [1.1, 2.2, 3.3] |
2 | Matrix (table of numbers) | m = [[1, 2, 3], [4, 5, 6], [7, 8, 9]] |
3 | 3-Tensor (cube of numbers) | t = [[[2], [4], [6]], [[8], [10], [12]], [[14], [16], [18]]] |
n | n-Tensor | .... |
Shape
The TensorFlow documentation uses three notational conventions to describe tensor dimensionality: rank, shape, and dimension number. The following table shows how these relate to one another:
Rank | Shape | Dimension number | Example |
---|---|---|---|
0 | [] | 0-D | A 0-D tensor. A scalar. |
1 | [D0] | 1-D | A 1-D tensor with shape [5]. |
2 | [D0, D1] | 2-D | A 2-D tensor with shape [3, 4]. |
3 | [D0, D1, D2] | 3-D | A 3-D tensor with shape [1, 4, 3]. |
n | [D0, D1, ... Dn] | n-D | A tensor with shape [D0, D1, ... Dn]. |
Shapes can be represented via Python lists / tuples of ints, or with the TensorShape
class.
Variables
Maintains state with Variables
.
Feeds
Uses feeds and fetches to get data into and out of arbitrary operations.
Fetches
Building the graph
To build a graph start with ops that do not need any input (source ops), such as Constant
, and pass their output to other ops that do computation.
The ops constructors in the Python library return objects that stand for the output of the constructed ops. You can pass these to other ops constructors to use as inputs.
The TensorFlow Python library has a default graph to which ops constructors add nodes. The default graph is sufficient for many applications. See the Graph class documentation for how to explicitly manage multiple graphs.
import tensorflow as tf
# Create a Constant op that produces a 1x2 matrix. The op is
# added as a node to the default graph.
#
# The value returned by the constructor represents the output
# of the Constant op.
matrix1 = tf.constant([[3., 3.]])
# Create another Constant that produces a 2x1 matrix.
matrix2 = tf.constant([[2.],[2.]])
# Create a Matmul op that takes 'matrix1' and 'matrix2' as inputs.
# The returned value, 'product', represents the result of the matrix
# multiplication.
product = tf.matmul(matrix1, matrix2)
The default graph now has three nodes: two constant()
ops and one matmul()
op. To actually multiply the matrices, and get the result of the multiplication, you must launch the graph in a session.
Launching the graph in a session
Launching follows construction. To launch a graph, create a Session
object. Without arguments the session constructor launches the default graph.
See the Session class for the complete session API.
# Launch the default graph.
sess = tf.Session()
# To run the matmul op we call the session 'run()' method, passing 'product'
# which represents the output of the matmul op. This indicates to the call
# that we want to get the output of the matmul op back.
#
# All inputs needed by the op are run automatically by the session. They
# typically are run in parallel.
#
# The call 'run(product)' thus causes the execution of three ops in the
# graph: the two constants and matmul.
#
# The output of the op is returned in 'result' as a numpy `ndarray` object.
result = sess.run(product)
print(result)
# ==> [[ 12.]]
# Close the Session when we're done.
sess.close()
Sessions should be closed to release resources. You can also enter a Session
with a "with" block. The Session
closes automatically at the end of the with
block.
with tf.Session() as sess:
result = sess.run([product])
print(result)
The TensorFlow implementation translates the graph definition into executable operations distributed across available compute resources, such as the CPU or one of your computer's GPU cards. In general you do not have to specify CPUs or GPUs explicitly. TensorFlow uses your first GPU, if you have one, for as many operations as possible.
If you have more than one GPU available on your machine, to use a GPU beyond the first you must assign ops to it explicitly. Use with...Device
statements to specify which CPU or GPU to use for operations:
with tf.Session() as sess:
with tf.device("/gpu:1"):
matrix1 = tf.constant([[3., 3.]])
matrix2 = tf.constant([[2.],[2.]])
product = tf.matmul(matrix1, matrix2)
...
Devices are specified with strings. The currently supported devices are:
-
"/cpu:0"
: The CPU of your machine. -
"/gpu:0"
: The GPU of your machine, if you have one. -
"/gpu:1"
: The second GPU of your machine, etc.
See Using GPUs for more information about GPUs and TensorFlow.
Launching the graph in a distributed session
To create a TensorFlow cluster, launch a TensorFlow server on each of the machines in the cluster. When you instantiate a Session in your client, you pass it the network location of one of the machines in the cluster:
with tf.Session("grpc://example.org:2222") as sess:
# Calls to sess.run(...) will be executed on the cluster.
...
This machine becomes the master for the session. The master distributes the graph across other machines in the cluster (workers), much as the local implementation distributes the graph across available compute resources within a machine.
You can use "with tf.device():" statements to directly specify workers for particular parts of the graph:
with tf.device("/job:ps/task:0"):
weights = tf.Variable(...)
biases = tf.Variable(...)
See the Distributed TensorFlow How To for more information about distributed sessions and clusters.
Interactive Usage
The Python examples in the documentation launch the graph with a Session
and use the Session.run()
method to execute operations.
For ease of use in interactive Python environments, such as IPython you can instead use theInteractiveSession
class, and the Tensor.eval()
and Operation.run()
methods. This avoids having to keep a variable holding the session.
# Enter an interactive TensorFlow Session.
import tensorflow as tf
sess = tf.InteractiveSession()
x = tf.Variable([1.0, 2.0])
a = tf.constant([3.0, 3.0])
# Initialize 'x' using the run() method of its initializer op.
x.initializer.run()
# Add an op to subtract 'a' from 'x'. Run it and print the result
sub = tf.sub(x, a)
print(sub.eval())
# ==> [-2. -1.]
# Close the Session when we're done.
sess.close()
Variables
Variables maintain state across executions of the graph. The following example shows a variable serving as a simple counter. See Variables for more details.
# Create a Variable, that will be initialized to the scalar value 0.
state = tf.Variable(0, name="counter")
# Create an Op to add one to `state`.
one = tf.constant(1)
new_value = tf.add(state, one)
update = tf.assign(state, new_value)
# Variables must be initialized by running an `init` Op after having
# launched the graph. We first have to add the `init` Op to the graph.
init_op = tf.initialize_all_variables()
# Launch the graph and run the ops.
with tf.Session() as sess:
# Run the 'init' op
sess.run(init_op)
# Print the initial value of 'state'
print(sess.run(state))
# Run the op that updates 'state' and print 'state'.
for _ in range(3):
sess.run(update)
print(sess.run(state))
# output:
# 0
# 1
# 2
# 3
The assign()
operation in this code is a part of the expression graph just like the add()
operation, so it does not actually perform the assignment until run()
executes the expression.
You typically represent the parameters of a statistical model as a set of Variables. For example, you would store the weights for a neural network as a tensor in a Variable. During training you update this tensor by running a training graph repeatedly.
Fetches
To fetch the outputs of operations, execute the graph with a run()
call on the Session
object and pass in the tensors to retrieve. In the previous example we fetched the single node state
, but you can also fetch multiple tensors:
input1 = tf.constant([3.0])
input2 = tf.constant([2.0])
input3 = tf.constant([5.0])
intermed = tf.add(input2, input3)
mul = tf.mul(input1, intermed)
with tf.Session() as sess:
result = sess.run([mul, intermed])
print(result)
# output:
# [array([ 21.], dtype=float32), array([ 7.], dtype=float32)]
All the ops needed to produce the values of the requested tensors are run once (not once per requested tensor).
Feeds
The examples above introduce tensors into the computation graph by storing them in Constants
andVariables
. TensorFlow also provides a feed mechanism for patching a tensor directly into any operation in the graph.
A feed temporarily replaces the output of an operation with a tensor value. You supply feed data as an argument to a run()
call. The feed is only used for the run call to which it is passed. The most common use case involves designating specific operations to be "feed" operations by using tf.placeholder() to create them:
input1 = tf.placeholder(tf.float32)
input2 = tf.placeholder(tf.float32)
output = tf.mul(input1, input2)
with tf.Session() as sess:
print(sess.run([output], feed_dict={input1:[7.], input2:[2.]}))
# output:
# [array([ 14.], dtype=float32)]
A placeholder()
operation generates an error if you do not supply a feed for it. See the MNIST fully-connected feed tutorial (source code) for a larger-scale example of feeds.
References
https://en.wikipedia.org/wiki/TensorFlow
https://research.googleblog.com/2015/11/tensorflow-googles-latest-machine_9.html
https://www.oreilly.com/learning/hello-tensorflow
http://wiki.jikexueyuan.com/project/tensorflow-zh/resources/glossary.html
https://hit-scir.gitbooks.io/neural-networks-and-deep-learning-zh_cn/content/chap3/c3s4.html
http://www.jianshu.com/p/c62fdd13561e
http://www.jianshu.com/p/7b95538d7cae
http://www.jianshu.com/p/1edde870eefe
https://www.tensorflow.org/versions/r0.9/how_tos/variables/index.html
https://morvanzhou.github.io/tutorials/machine-learning/tensorflow/2-2-example2/
http://list.youku.com/albumlist/show?id=27327189&ascending=1&page=1
http://hp.stuhome.net/index.php/2016/07/06/tensorflow-learning-paths/
http://neuralnetworksanddeeplearning.com/
http://colah.github.io/posts/2014-07-Conv-Nets-Modular/
http://deeplearning.net/tutorial/
http://learningtensorflow.com/getting_started/
http://jorditorres.org/first-contact-with-tensorflow/
https://www.zhihu.com/question/49909565
https://zh.wikipedia.org/wiki/%E7%BA%BF%E6%80%A7%E6%95%B4%E6%B5%81%E5%87%BD%E6%95%B0
https://cs287.github.io/Lectures/slides/lecture7-wc.pdf
推荐阅读
-
win10安装tensorflow-gpu1.13.1+cuda10.0+cudnn7.3.1
-
TensorFlow在MAC环境下的安装及环境搭建
-
tensorflow教程推荐(新手必知的tensorflow基础知识)
-
详细介绍如何用TensorFlow实现yolo v3 macOS
-
windows10下安装TensorFlow Object Detection API的步骤
-
浅谈Tensorflow模型的保存与恢复加载
-
基于神经网络的人脸识别tensorflow(数据的存储与加载)
-
TensorFlow进阶项目实战
-
在python下使用tensorflow判断是否存在文件夹的实例
-
2-1 Tensorflow --创建图、启动图