Tensor - Data Representation in Neural Networks

Tensors are the fundamental data structure used by all machine and deep learning algorithms. A tensor can be a generic structure that can be used for storing, representing, and changing data.

All inputs, outputs and transformations in deep learning are represented through tensors only. Tensors are widely used in many different fields, including machine learning, data analysis, and image processing.

Tensors are mathematical objects that generalize matrices to higher dimensions. Tensors can be represented as an array data structure.

Deep Learning: In the general case, an array of numbers arranged on a regular grid with a variable number of axes is known as a tensor.

1. What are Tensors?

A tensor is just a container for data, typically numerical data. It is, therefore, a container for numbers. Tensors are a generalization of matrices to any number of dimensions. You may already be familiar with matrices, which are 2D tensors (note that in the context of tensors, a dimension is often called an axis).
张量是多维数组。

Tensors are mathematical objects that describe linear relationships between sets of multidimensional data. They are a generalization of scalars, vectors, and matrices, which are all types of tensors.

Tensors are like matrices, but they can have more than two dimensions. For example, a 3D tensor can be thought of as a cube of numbers, with each number representing a different element in the tensor. A 4D tensor can be thought of as a stack of cubes, and so on.

Graphically, it is easiest to represent tensors by using some kind of geometric shape (for example, a circle) while modes are represented by lines (often called legs).
在图形上，使用某种几何形状 (例如，圆形) 来表示张量是最简单的，而模态则由线 (通常称为腿) 表示。

A scalar, vector, and matrix can all be represented as tensors in a more generalized fashion. An n-dimensional matrix serves as the definition of a tensor. A scalar is a zero-dimensional tensor (i.e., a single number), a vector is a one-dimensional tensor, a matrix is a two-dimensional tensor, a cube is a three-dimensional tensor, etc. The rank of a tensor is another name for a matrix’s dimension.
scalar, vector, and matrix 都可以更通用的方式表示为张量。N 维矩阵用作张量的定义。标量是零维张量 (即单个数)，向量是一维张量，矩阵是二维张量，立方体是三维张量等。张量的秩是矩阵维度的另一个名称。

Tensor is nothing but a container of data. In tensor terms, a matrix is a two dimensional (2D) tensor. In a similar way, a vector is a one-dimensional tensor whereas a scalar is a zero-dimensional tensor.

Scalars are single numbers, such as 3 or 5. Vectors are arrays of numbers, such as [1, 2, 3] or [4, 5, 6]. Matrices are 2D arrays of numbers, such as [[1, 2], [3, 4]] or [[5, 6], [7, 8]].

• Rank or Number of axes

A matrix contains two axes, while a 3D tensor possesses three. In Python libraries like Numpy, this is additionally referred to as the tensor’s ndim.
矩阵包含两个轴，而 3D 张量具有三个轴。在像 Numpy 这样的 Python 库中，这也被称为张量的 ndim。

The term rank is used to define the dimension or the indexes of the tensor.

Number of axes of a tensor is also termed as the rank of the tensor. Simply speaking, the axes or rank of the tensor represented as an array is number of indices that will be required to access a specific value in the multi-dimensional array aka tensor. Accessing a specific element in a tensor is also called as tensor slicing. The first axis of the tensor is also called as a sample axis.

• Shape

The number of dimensions the tensor contains across each axis is specified by a tuple of integers. For instance, the 3D tensor example has shape (2, 3, 4) while the matrix example has shape (2, 5). A scalar has an empty shape as (), but a vector has a shape with a single element, like (5,).
张量在每个轴上包含的维数由整数元组指定。

A vector has only one axis / dimension so the shape is just a single element. The vector we used here as an example has 5 elements so its shape is (5,).

• Date type

The format of the data which makes up the tensor. A char tensor might appear in exceptional cases. Due to the string’s changeable duration and the fact that tensors reside well before shared memory sections, string tensors are not present in Numpy (or in the majority of other libraries).
Numpy (或大多数其他库) 中不存在字符串张量。

1.1 Scalar data - 0D tensor

The term scalar (also known as scalar-tensor, 0-dimensional tensor, or 0D tensor) refers to a tensor that only holds a single number. The ndim feature of a Numpy tensor can be used to indicate the number of axes, a scalar-tensor has no axes (ndim == 0). A tensor’s rank is another name for the number of its axes.
标量。

These scalar data has rank zero as they have zero axes. Python’s ndim attribute can display the number of axes of any data structure.

(base) yongqiang@yongqiang:~$ python
Python 3.9.5 (default, Jun  4 2021, 12:28:51)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> a = np.array(9)
>>> a
array(9)
>>> a.ndim
0
>>> a.shape
()
>>>
>>> exit()
(base) yongqiang@yongqiang:~$

1.2 Vector data - 1D tensor

A vector, often known as a 1D tensor, is a collection of numbers. It is claimed that a 1D tensor has just one axis.
向量。

(base) yongqiang@yongqiang:~$ python
Python 3.9.5 (default, Jun  4 2021, 12:28:51)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> b = np.array([8, 6, 4, 2, 0])
>>> b
array([8, 6, 4, 2, 0])
>>> b.ndim
1
>>> b.shape
(5,)
>>>
>>> exit()
(base) yongqiang@yongqiang:~$

This vector is referred to as a 5D vector since it has five elements. A 5D tensor is not the same as a 5D vector. A 5D tensor will have five axes, whereas a 5D vector has just only one axis and five dimensions along it (and may have any number of dimensions along each axis). Dimensionality can refer to the number of axes in a tensor, such as a 5D tensor, or the number of entries along a particular axis, as in the case of our 5D vector. Although using the ambiguous notation 5D tensor is widespread, using a tensor of rank 5 (the rank of a tensor being the number of axes) is mathematically more accurate in the latter situation.
这个向量被称为 5D vector，因为它有五个元素。5D tensor 与 5D vector 不同。一个 5D tensor 有五个轴，而一个 5D vector 只有一个轴和沿它的五个维度 (并且可以沿每个轴有任意数量的维度)。维数可以指张量中的轴数，例如 5D tensor，或沿特定轴的条目数，如我们的 5D vector。尽管使用模棱两可的符号 5D tensor 很普遍，但在后一种情况下，使用阶数为 5 的张量 (张量的阶数是轴的数量) 在数学上更准确。

1.3 Matrix data - 2D tensor (samples, features) or (rows, columns)

A matrix, or 2D tensor, is a collection of vectors. Two axes constitute a matrix (often referred to as rows and columns). A matrix has rows and columns hence two axes and rank is two.
矩阵。

A batch of data in a dataset will be stored as a 2D tensor (i.e., an array of vectors), in which the first axis is the samples axis and the second axis is the features axis. Each individual data point in such a dataset is stored as a vector.
数据集中的一批数据将存储为二维张量 (即向量数组)，其中第一个轴是 samples axis，第二个轴是 features axis。这种数据集中的每个单独的数据点都存储为一个向量。samples 代表样本，features 代表特征。

(base) yongqiang@yongqiang:~$ python
Python 3.9.5 (default, Jun  4 2021, 12:28:51)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> c = np.array([[8, 6, 4, 2, 0], [1, 3, 5, 7, 9]])
>>> c
array([[8, 6, 4, 2, 0],[1, 3, 5, 7, 9]])
>>> c.ndim
2
>>> c.shape
(2, 5)
>>>
>>> exit()
(base) yongqiang@yongqiang:~$

Rows and columns are used to describe the elements from the first and second axes. The first row of c in the example is [8, 6, 4, 2, 0], while the first column is [8, 1].

A = [[1, 2, 3],[4, 5, 6],[7, 8, 9]]
Tensor A has rank = 2; i.e., its a 2-D tensor, i.e., the index =2 so the no of axes = 2
Each element along the first axis represents -> [array]
A[0] = [1,2,3]
A[1] = [4,5,6]
A[2] = [7,8,9]
Each element along the second axis represents-> [numbers/data]
A[0][0] = 1
A[0][1] = 2
A[0][2] = 3
.
.

1.4 Time series data or sequence data - 3D tensor (samples, timesteps, features / characteristics)

A batch of data will be encoded as a 3D tensor because each instance can be represented as a series of vectors (a 2D tensor).
一批数据将被编码为 3D 张量，因为每个实例都可以表示为一系列向量 (2D 张量)。

The time axis has always been the second axis (axis of index 1) by convention.
按照惯例，时间轴始终是第二个轴 (索引 1 的轴)。

These matrices can be combined into a new array to create a 3D tensor, which can be seen as a cube of integers. A 4D tensor can be produced by stacking 3D tensors in an array, and so on. In deep learning, you typically work with tensors that range from 0 to 4D, though if you’re processing video data, you might go as high as 5D.

cube [kjuːb]：n. 立方体，立方形，立方形的东西，三次幂 v. 求 ... 的立方，把 (食物) 切成小方块
imperative [ɪm'perətɪv]：adj. 重要紧急的，迫切的，急需处理的，表示权威的 n. 重要紧急的事，必要的事，祈使语气，祈使语气动词
sample_size = samples

(base) yongqiang@yongqiang:~$ python
Python 3.9.5 (default, Jun  4 2021, 12:28:51)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> d = np.arange(24)
>>> d
array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16,17, 18, 19, 20, 21, 22, 23])
>>>
>>> d = d.reshape(2, 3, 4)
>>> d
array([[[ 0,  1,  2,  3],[ 4,  5,  6,  7],[ 8,  9, 10, 11]],[[12, 13, 14, 15],[16, 17, 18, 19],[20, 21, 22, 23]]])
>>> d.ndim
3
>>> d.shape
(2, 3, 4)
>>>
>>> exit()
(base) yongqiang@yongqiang:~$

1.5 Image data - 4D tensor (samples, height, width, channels) or (samples, channels, height, width)

Height, width, and colour depth are the three dimensions that most images have. By definition, image tensors are always 3D, with a one-dimensional colour channel for grayscale images. Even though grayscale images (like our MNIST digits) only have a single colour channel and may therefore be stored in 2D tensors. Thus, a tensor of shape (32, 64, 64, 1) might be used to save a batch of 32 grayscale photos of size 64 x 64, while a tensor of shape (32, 64, 64, 1) could be used to store a batch of 32 colour images (32, 64, 64, 3).

The channels-last format (used by TensorFlow) and the channels-first format are the two standards for the shapes of image tensors (used by Theano). The colour-depth axis is located at the end of the list of dimensions in the Google TensorFlow: (samples, height, width, colour-depth). The batch axis is placed first, followed by the samples, colour-depth, height, and width axes by Theano. Both file types are supported by the Keras framework.

1.6 Video data - 5D tensor (samples, frames, height, width, channels) or (samples, frames, channels, height, width)

A video could be viewed as a set of coloured images called frames. A batch of various movies can be saved in a 5D tensor of shape (samples, frames, height, width, and colour-depth) since each frame can be kept in a 3D tensor (height, width, and colour-depth). A series of frames can also be saved in a 4D tensor (frames, height, width, and colour-depth).

For instance, 240 frames would be present in a 60 seconds, 144 x 256 YouTube video clip sampled at 4 frames per second. Four of these video clips would be saved in a tensor shape as a batch (4, 240, 144, 256, 3). There are 106,168,320 values in all.

frames 代表帧，每一帧都是一张彩色图像。

2. Introduction to TensorFlow Tensors

https://www.tensorflow.org/guide/tensor

import tensorflow as tf
import numpy as np

2022-12-14 03:00:09.384365: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory
2022-12-14 03:00:09.384454: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory
2022-12-14 03:00:09.384463: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.

Tensors are multi-dimensional arrays with a uniform type (called a dtype). You can see all supported dtypes at tf.dtypes.DType.
张量是具有统一类型 (称为 dtype) 的多维数组。您可以在 tf.dtypes.DType 中查看所有支持的 dtypes。

If you’re familiar with NumPy, tensors are (kind of) like np.arrays.
如果您熟悉 NumPy，就会知道张量与 np.arrays 有一定的相似性。

All tensors are immutable like Python numbers and strings: you can never update the contents of a tensor, only create a new one.
就像 Python 数值和字符串一样，所有张量都是不可变的：永远无法更新张量的内容，只能创建新的张量。

immutable [ɪˈmjuːtəb(ə)l]：adj. 不可改变的，永恒不变的

2.1 Basics

First, create some basic tensors.
Here is a scalar or rank-0 tensor. A scalar contains a single value, and no axes.
下面是一个标量 (或称 0 秩张量)。标量包含单个值，但没有轴。

# This will be an int32 tensor by default; see "dtypes" below.
rank_0_tensor = tf.constant(4)
print(rank_0_tensor)tf.Tensor(4, shape=(), dtype=int32)

A vector or rank-1 tensor is like a list of values. A vector has one axis:
向量 (或称 1 秩张量) 就像一个值列表。向量有 1 个轴：

# Let's make this a float tensor.
rank_1_tensor = tf.constant([2.0, 3.0, 4.0])
print(rank_1_tensor)tf.Tensor([2. 3. 4.], shape=(3,), dtype=float32)

A matrix or rank-2 tensor has two axes:
矩阵 (或称 2 秩张量) 有 2 个轴：

# If you want to be specific, you can set the dtype (see below) at creation time
rank_2_tensor = tf.constant([[1, 2],[3, 4],[5, 6]], dtype=tf.float16)
print(rank_2_tensor)tf.Tensor(
[[1. 2.][3. 4.][5. 6.]], shape=(3, 2), dtype=float16)

Tensors may have more axes; here is a tensor with three axes:
张量的轴可能更多，下面是一个包含 3 个轴的张量：

# There can be an arbitrary number of axes (sometimes called "dimensions")
rank_3_tensor = tf.constant([[[0, 1, 2, 3, 4],[5, 6, 7, 8, 9]],[[10, 11, 12, 13, 14],[15, 16, 17, 18, 19]],[[20, 21, 22, 23, 24],[25, 26, 27, 28, 29]],])print(rank_3_tensor)tf.Tensor(
[[[ 0  1  2  3  4][ 5  6  7  8  9]][[10 11 12 13 14][15 16 17 18 19]][[20 21 22 23 24][25 26 27 28 29]]], shape=(3, 2, 5), dtype=int32)

There are many ways you might visualize a tensor with more than two axes.
对于包含 2 个以上的轴的张量，您可以通过多种方式加以呈现。

You can convert a tensor to a NumPy array either using np.array or the tensor.numpy method:
通过使用 np.array 或 tensor.numpy 方法，您可以将张量转换为 NumPy 数组：

np.array(rank_2_tensor)array([[1., 2.],[3., 4.],[5., 6.]], dtype=float16)

rank_2_tensor.numpy()array([[1., 2.],[3., 4.],[5., 6.]], dtype=float16)

Tensors often contain floats and ints, but have many other types, including:
张量通常包含浮点型和整型数据，但是还有许多其他数据类型，包括：

• complex numbers
• strings

The base tf.Tensor class requires tensors to be rectangular - that is, along each axis, every element is the same size. However, there are specialized types of tensors that can handle different shapes:
tf.Tensor 基类要求张量是矩形 - 也就是说，每个轴上的每一个元素大小相同。但是，张量有可以处理不同形状的特殊类型。

• Ragged tensors (不规则张量)
• Sparse tensors (稀疏张量)

You can do basic math on tensors, including addition, element-wise multiplication, and matrix multiplication.
您可以对张量执行基本数学运算，包括加法、逐元素乘法和矩阵乘法。

a = tf.constant([[1, 2],[3, 4]])
b = tf.constant([[1, 1],[1, 1]]) # Could have also said `tf.ones([2,2])`print(tf.add(a, b), "\n")
print(tf.multiply(a, b), "\n")
print(tf.matmul(a, b), "\n")tf.Tensor(
[[2 3][4 5]], shape=(2, 2), dtype=int32) tf.Tensor(
[[1 2][3 4]], shape=(2, 2), dtype=int32) tf.Tensor(
[[3 3][7 7]], shape=(2, 2), dtype=int32)

print(a + b, "\n") # element-wise addition
print(a * b, "\n") # element-wise multiplication
print(a @ b, "\n") # matrix multiplicationtf.Tensor(
[[2 3][4 5]], shape=(2, 2), dtype=int32) tf.Tensor(
[[1 2][3 4]], shape=(2, 2), dtype=int32) tf.Tensor(
[[3 3][7 7]], shape=(2, 2), dtype=int32)

Tensors are used in all kinds of operations (or “Ops”).

c = tf.constant([[4.0, 5.0], [10.0, 1.0]])# Find the largest value
print(tf.reduce_max(c))
# Find the index of the largest value
print(tf.math.argmax(c))
# Compute the softmax
print(tf.nn.softmax(c))tf.Tensor(10.0, shape=(), dtype=float32)
tf.Tensor([1 0], shape=(2,), dtype=int64)
tf.Tensor(
[[2.6894143e-01 7.3105854e-01][9.9987662e-01 1.2339458e-04]], shape=(2, 2), dtype=float32)

Note: Typically, anywhere a TensorFlow function expects a Tensor as input, the function will also accept anything that can be converted to a Tensor using tf.convert_to_tensor. See below for an example.
通常，在 TensorFlow 函数需要 Tensor 作为输入的任何地方，该函数也将接受可使用 tf.convert_to_tensor 转换为 Tensor 的任何内容。请参见下面的示例。

tf.convert_to_tensor([1,2,3])<tf.Tensor: shape=(3,), dtype=int32, numpy=array([1, 2, 3], dtype=int32)>

tf.reduce_max([1,2,3])<tf.Tensor: shape=(), dtype=int32, numpy=3>

tf.reduce_max(np.array([1,2,3]))<tf.Tensor: shape=(), dtype=int64, numpy=3>

2.2 Shapes

Tensors have shapes.

• Shape: The length (number of elements) of each of the axes of a tensor.
  张量的每个轴的长度 / 元素数量。
• Rank: Number of tensor axes. A scalar has rank 0, a vector has rank 1, a matrix is rank 2.
  张量轴数。标量的秩为 0，向量的秩为 1，矩阵的秩为 2。
• Axis or Dimension: A particular dimension of a tensor.
  张量的一个特殊维度。
• Size: The total number of items in the tensor, the product of the shape vector’s elements.
  张量的总项数，即形状矢量元素的乘积。

Note: Although you may see reference to a tensor of two dimensions, a rank-2 tensor does not usually describe a 2D space.
虽然您可能会看到 tensor of two dimensions 之类的表述，但 rank-2 tensor 通常并不是用来描述二维空间。

Tensors and tf.TensorShape objects have convenient properties for accessing these:
张量和 tf.TensorShape 对象提供了方便的属性来访问：

rank_4_tensor = tf.zeros([3, 2, 4, 5])

print("Type of every element:", rank_4_tensor.dtype)
print("Number of axes:", rank_4_tensor.ndim)
print("Shape of tensor:", rank_4_tensor.shape)
print("Elements along axis 0 of tensor:", rank_4_tensor.shape[0])
print("Elements along the last axis of tensor:", rank_4_tensor.shape[-1])
print("Total number of elements (3*2*4*5): ", tf.size(rank_4_tensor).numpy())Type of every element: <dtype: 'float32'>
Number of axes: 4
Shape of tensor: (3, 2, 4, 5)
Elements along axis 0 of tensor: 3
Elements along the last axis of tensor: 5
Total number of elements (3*2*4*5):  120

But note that the Tensor.ndim and Tensor.shape attributes don’t return Tensor objects. If you need a Tensor use the tf.rank or tf.shape function. This difference is subtle, but it can be important when building graphs (later).
Tensor.ndim and Tensor.shape 特性不返回 Tensor 对象。如果您需要 Tensor，请使用 tf.rank 或 tf.shape 函数。这种差异不易察觉，但在构建计算图时可能非常重要。

tf.rank(rank_4_tensor)<tf.Tensor: shape=(), dtype=int32, numpy=4>

tf.shape(rank_4_tensor)<tf.Tensor: shape=(4,), dtype=int32, numpy=array([3, 2, 4, 5], dtype=int32)>

subtle ['sʌt(ə)l]：adj. 不易察觉的，不明显的，微妙的，机智的

While axes are often referred to by their indices, you should always keep track of the meaning of each. Often axes are ordered from global to local: The batch axis first, followed by spatial dimensions, and features for each location last. This way feature vectors are contiguous regions of memory.
通常用索引来指代轴，但是您始终要记住每个轴的含义。轴一般按照从全局到局部的顺序进行排序：首先是 batch axis，随后是空间维度，最后是每个位置的特征。这样，在内存中，特征向量就会位于连续的区域。

2.3 Indexing (索引)

2.3.1 Single-axis indexing (单轴索引)

TensorFlow follows standard Python indexing rules, similar to indexing a list or a string in Python, and the basic rules for NumPy indexing.
TensorFlow 遵循标准 Python 索引规则 (类似于在 Python 中为列表或字符串编制索引) 以及 NumPy 索引的基本规则。https://docs.python.org/3/tutorial/introduction.html

• indexes start at 0 (索引从 0 开始编制)
• negative indices count backwards from the end (负索引表示按倒序编制索引)
• colons, :, are used for slices: start:stop:step (冒号 : 用于切片 start:stop:step)

rank_1_tensor = tf.constant([0, 1, 1, 2, 3, 5, 8, 13, 21, 34])
print(rank_1_tensor.numpy())[ 0  1  1  2  3  5  8 13 21 34]

Indexing with a scalar removes the axis:
使用标量编制索引会移除轴：

print("First:", rank_1_tensor[0].numpy())
print("Second:", rank_1_tensor[1].numpy())
print("Last:", rank_1_tensor[-1].numpy())First: 0
Second: 1
Last: 34

Indexing with a : slice keeps the axis:
使用 : 切片编制索引会保留轴：

print("Everything:", rank_1_tensor[:].numpy())
print("Before 4:", rank_1_tensor[:4].numpy())
print("From 4 to the end:", rank_1_tensor[4:].numpy())
print("From 2, before 7:", rank_1_tensor[2:7].numpy())
print("Every other item:", rank_1_tensor[::2].numpy())
print("Reversed:", rank_1_tensor[::-1].numpy())Everything: [ 0  1  1  2  3  5  8 13 21 34]
Before 4: [0 1 1 2]
From 4 to the end: [ 3  5  8 13 21 34]
From 2, before 7: [1 2 3 5 8]
Every other item: [ 0  1  3  8 21]
Reversed: [34 21 13  8  5  3  2  1  1  0]

2.3.2 Multi-axis indexing (多轴索引)

Higher rank tensors are indexed by passing multiple indices.
更高阶的张量通过传递多个索引来编制索引。

The exact same rules as in the single-axis case apply to each axis independently.
对于高秩张量的每个单独的轴，遵循与单轴情形完全相同的规则。

print(rank_2_tensor.numpy())[[1. 2.][3. 4.][5. 6.]]

Passing an integer for each index, the result is a scalar.
为每个索引传递一个整数，结果是一个标量。

# Pull out a single value from a 2-rank tensor
print(rank_2_tensor[1, 1].numpy())4.0

You can index using any combination of integers and slices:
您可以使用整数与切片的任意组合编制索引：

# Get row and column tensors
print("Second row:", rank_2_tensor[1, :].numpy())
print("Second column:", rank_2_tensor[:, 1].numpy())
print("Last row:", rank_2_tensor[-1, :].numpy())
print("First item in last column:", rank_2_tensor[0, -1].numpy())
print("Skip the first row:")
print(rank_2_tensor[1:, :].numpy(), "\n")Second row: [3. 4.]
Second column: [2. 4. 6.]
Last row: [5. 6.]
First item in last column: 2.0
Skip the first row:
[[3. 4.][5. 6.]]

Here is an example with a 3-axis tensor:
下面是一个 3 轴张量的示例：

print(rank_3_tensor[:, :, 4])tf.Tensor(
[[ 4  9][14 19][24 29]], shape=(3, 2), dtype=int32)

Read the tensor slicing guide to learn how you can apply indexing to manipulate individual elements in your tensors.
https://www.tensorflow.org/guide/tensor_slicing
应用索引编制以操作张量中的各个元素。

2.4 Manipulating Shapes

Reshaping a tensor is of great utility.
改变张量的形状很有用。

# Shape returns a `TensorShape` object that shows the size along each axis
x = tf.constant([[1], [2], [3]])
print(x.shape)(3, 1)

# You can convert this object into a Python list, too
print(x.shape.as_list())[3, 1]

You can reshape a tensor into a new shape. The tf.reshape operation is fast and cheap as the underlying data does not need to be duplicated.
通过 reshape 可以改变张量的形状。tf.reshape 的速度很快，资源消耗很低，因为不需要复制底层数据。

# You can reshape a tensor to a new shape.
# Note that you're passing in a list
reshaped = tf.reshape(x, [1, 3])
print(x.shape)
print(reshaped.shape)(3, 1)
(1, 3)

The data maintains its layout in memory and a new tensor is created, with the requested shape, pointing to the same data. TensorFlow uses C-style row-major memory ordering, where incrementing the rightmost index corresponds to a single step in memory.
数据在内存中的布局保持不变，同时使用请求的形状创建一个指向同一数据的新张量。TensorFlow 采用 C 样式的行优先内存访问顺序，即最右侧的索引值递增对应于内存中的单步位移。

print(rank_3_tensor)tf.Tensor(
[[[ 0  1  2  3  4][ 5  6  7  8  9]][[10 11 12 13 14][15 16 17 18 19]][[20 21 22 23 24][25 26 27 28 29]]], shape=(3, 2, 5), dtype=int32)

If you flatten a tensor you can see what order it is laid out in memory.
如果您展平张量，则可以看到它在内存中的排列顺序。

# A `-1` passed in the `shape` argument says "Whatever fits".
print(tf.reshape(rank_3_tensor, [-1]))tf.Tensor(
[ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 2324 25 26 27 28 29], shape=(30,), dtype=int32)

Typically the only reasonable use of tf.reshape is to combine or split adjacent axes (or add/remove 1s).
一般来说，tf.reshape 唯一合理的用途是用于合并或拆分相邻轴 (或添加/移除 1)。

For this 3 x 2 x 5 tensor, reshaping to (3 x 2) x 5 or 3 x (2 x 5) are both reasonable things to do, as the slices do not mix:
对于 3 x 2 x 5 张量，重构为 (3 x 2) x 5 或 3 x (2 x 5) 都合理，因为切片不会混淆：

print(tf.reshape(rank_3_tensor, [3*2, 5]), "\n")
print(tf.reshape(rank_3_tensor, [3, -1]))tf.Tensor(
[[ 0  1  2  3  4][ 5  6  7  8  9][10 11 12 13 14][15 16 17 18 19][20 21 22 23 24][25 26 27 28 29]], shape=(6, 5), dtype=int32) tf.Tensor(
[[ 0  1  2  3  4  5  6  7  8  9][10 11 12 13 14 15 16 17 18 19][20 21 22 23 24 25 26 27 28 29]], shape=(3, 10), dtype=int32)

Reshaping will “work” for any new shape with the same total number of elements, but it will not do anything useful if you do not respect the order of the axes.
reshape 可以处理总元素个数相同的任何新形状，但是如果不遵从轴的顺序，则不会发挥任何作用。

Swapping axes in tf.reshape does not work; you need tf.transpose for that.
利用 tf.reshape 无法实现轴的交换，要交换轴，您需要使用 tf.transpose。

# Bad examples: don't do this# You can't reorder axes with reshape.
print(tf.reshape(rank_3_tensor, [2, 3, 5]), "\n") # This is a mess
print(tf.reshape(rank_3_tensor, [5, 6]), "\n")# This doesn't work at all
try:tf.reshape(rank_3_tensor, [7, -1])
except Exception as e:print(f"{type(e).__name__}: {e}")tf.Tensor(
[[[ 0  1  2  3  4][ 5  6  7  8  9][10 11 12 13 14]][[15 16 17 18 19][20 21 22 23 24][25 26 27 28 29]]], shape=(2, 3, 5), dtype=int32) tf.Tensor(
[[ 0  1  2  3  4  5][ 6  7  8  9 10 11][12 13 14 15 16 17][18 19 20 21 22 23][24 25 26 27 28 29]], shape=(5, 6), dtype=int32) InvalidArgumentError: Input to reshape is a tensor with 30 values, but the requested shape requires a multiple of 7 [Op:Reshape]

mess [mes]：n. 混乱，餐厅，杂乱，肮脏 v. 使不整洁，弄脏，弄乱，随地便溺

You may run across not-fully-specified shapes. Either the shape contains a None (an axis-length is unknown) or the whole shape is None (the rank of the tensor is unknown).
您可能会遇到非完全指定的形状。要么是形状包含 None (轴长度未知)，要么是整个形状为 None (张量的秩未知)。

Except for tf.RaggedTensor, such shapes will only occur in the context of TensorFlow’s symbolic, graph-building APIs:
除了 tf.RaggedTensor 外，这种情况只会在 TensorFlow 的符号化计算图构建 API 环境中出现：

• tf.function
• The keras functional API.

rag [ræɡ]：n. 抹布，破布，质量低劣的报纸，小报 v. 嘲笑，捉弄
symbolic [sɪmˈbɒlɪk]：adj. 使用象征的，作为象征的，象征性的

2.5 More on DTypes

To inspect a tf.Tensor’s data type use the Tensor.dtype property.
使用 Tensor.dtype 属性可以检查 tf.Tensor 的数据类型。

When creating a tf.Tensor from a Python object you may optionally specify the datatype.
从 Python 对象创建 tf.Tensor 时，您可以选择指定数据类型。

If you don’t, TensorFlow chooses a datatype that can represent your data. TensorFlow converts Python integers to tf.int32 and Python floating point numbers to tf.float32. Otherwise TensorFlow uses the same rules NumPy uses when converting to arrays.
如果不指定，TensorFlow 会选择一个可以表示您的数据的数据类型。TensorFlow 将 Python 整数转换为 tf.int32，将 Python 浮点数转换为 tf.float32。另外，当转换为数组时，TensorFlow 会采用与 NumPy 相同的规则。

You can cast from type to type.
数据类型可以相互转换。

the_f64_tensor = tf.constant([2.2, 3.3, 4.4], dtype=tf.float64)
the_f16_tensor = tf.cast(the_f64_tensor, dtype=tf.float16)
# Now, cast to an uint8 and lose the decimal precision
the_u8_tensor = tf.cast(the_f16_tensor, dtype=tf.uint8)
print(the_u8_tensor)tf.Tensor([2 3 4], shape=(3,), dtype=uint8)

2.6 Broadcasting (广播)

Broadcasting is a concept borrowed from the equivalent feature in NumPy. In short, under certain conditions, smaller tensors are “stretched” automatically to fit larger tensors when running combined operations on them.
广播是从 NumPy 中的等效功能借用的一个概念。简而言之，在一定条件下，对一组张量执行组合运算时，为了适应大张量，会对小张量进行扩展。

The simplest and most common case is when you attempt to multiply or add a tensor to a scalar. In that case, the scalar is broadcast to be the same shape as the other argument.
最简单和最常见的例子是尝试将张量与标量相乘或相加。在这种情况下会对标量进行广播，使其变成与其他参数相同的形状。

x = tf.constant([1, 2, 3])y = tf.constant(2)
z = tf.constant([2, 2, 2])
# All of these are the same computation
print(tf.multiply(x, 2))
print(x * y)
print(x * z)tf.Tensor([2 4 6], shape=(3,), dtype=int32)
tf.Tensor([2 4 6], shape=(3,), dtype=int32)
tf.Tensor([2 4 6], shape=(3,), dtype=int32)

Likewise, axes with length 1 can be stretched out to match the other arguments. Both arguments can be stretched in the same computation.
同样，可以扩展长度为 1 的轴，使其匹配其他参数。在同一个计算中可以同时扩展两个参数。

In this case a 3 x 1 matrix is element-wise multiplied by a 1 x 4 matrix to produce a 3 x 4 matrix. Note how the leading 1 is optional: The shape of y is [4].
在本例中，一个 3 x 1 的矩阵与一个 1 x 4 进行元素级乘法运算，从而产生一个 3 x 4 的矩阵。注意前导 1 是可选的：y 的形状是 [4]。

# These are the same computations
x = tf.reshape(x, [3,1])
y = tf.range(1, 5)
print(x, "\n")
print(y, "\n")
print(tf.multiply(x, y))tf.Tensor(
[[1][2][3]], shape=(3, 1), dtype=int32) tf.Tensor([1 2 3 4], shape=(4,), dtype=int32) tf.Tensor(
[[ 1  2  3  4][ 2  4  6  8][ 3  6  9 12]], shape=(3, 4), dtype=int32)

Here is the same operation without broadcasting:
下面是不使用广播的同一运算：

x_stretch = tf.constant([[1, 1, 1, 1],[2, 2, 2, 2],[3, 3, 3, 3]])y_stretch = tf.constant([[1, 2, 3, 4],[1, 2, 3, 4],[1, 2, 3, 4]])print(x_stretch * y_stretch)  # Again, operator overloadingtf.Tensor(
[[ 1  2  3  4][ 2  4  6  8][ 3  6  9 12]], shape=(3, 4), dtype=int32)

Most of the time, broadcasting is both time and space efficient, as the broadcast operation never materializes the expanded tensors in memory.
在大多数情况下，广播的时间和空间效率更高，因为广播运算不会在内存中具体化扩展的张量。

You see what broadcasting looks like using tf.broadcast_to.
使用 tf.broadcast_to 可以了解广播的运算方式。

print(tf.broadcast_to(tf.constant([1, 2, 3]), [3, 3]))tf.Tensor(
[[1 2 3][1 2 3][1 2 3]], shape=(3, 3), dtype=int32)

Unlike a mathematical op, for example, broadcast_to does nothing special to save memory. Here, you are materializing the tensor.
与数学运算不同，broadcast_to 并不会节省内存。在这个示例中，您将具体化张量。

It can get even more complicated. This section of Jake VanderPlas’s book Python Data Science Handbook shows more broadcasting tricks (again in NumPy).
https://jakevdp.github.io/PythonDataScienceHandbook/02.05-computation-on-arrays-broadcasting.html

materialize [məˈtɪəriəlaɪz]：v. 实现，发生，成为现实，突然显现

2.7 tf.convert_to_tensor

Most ops, like tf.matmul and tf.reshape take arguments of class tf.Tensor. However, you’ll notice in the above case, Python objects shaped like tensors are accepted.
大部分运算 (如 tf.matmul 和 tf.reshape) 会使用 tf.Tensor 类的参数。不过，在上面的示例中，您会发现形状类似于张量的 Python 对象也可以接受。

Most, but not all, ops call convert_to_tensor on non-tensor arguments. There is a registry of conversions, and most object classes like NumPy’s ndarray, TensorShape, Python lists, and tf.Variable will all convert automatically.
大部分 (但并非全部) 运算会在非张量参数上调用 convert_to_tensor。我们提供了一个转换注册表，大多数对象类 (如 NumPy 的 ndarray、TensorShape、Python 列表和 tf.Variable) 都可以自动转换。

See tf.register_tensor_conversion_function for more details, and if you have your own type you’d like to automatically convert to a tensor.
如果您有自己的类型，则可能希望自动转换为张量。

2.8 Ragged Tensors (不规则张量)

A tensor with variable numbers of elements along some axis is called “ragged”. Use tf.ragged.RaggedTensor for ragged data.
如果张量的某个轴上的元素个数可变，则称为不规则张量。对于不规则数据，请使用 tf.ragged.RaggedTensor。

For example, this cannot be represented as a regular tensor:
下面的例子无法用规则张量表示

ragged_list = [[0, 1, 2, 3],[4, 5],[6, 7, 8],[9]]try:tensor = tf.constant(ragged_list)
except Exception as e:print(f"{type(e).__name__}: {e}")ValueError: Can't convert non-rectangular Python sequence to Tensor.

Instead create a tf.RaggedTensor using tf.ragged.constant:

ragged_tensor = tf.ragged.constant(ragged_list)
print(ragged_tensor)<tf.RaggedTensor [[0, 1, 2, 3], [4, 5], [6, 7, 8], [9]]>

The shape of a tf.RaggedTensor will contain some axes with unknown lengths:

print(ragged_tensor.shape)(4, None)

2.9 String tensors (字符串张量)

tf.string is a dtype, which is to say you can represent data as strings (variable-length byte arrays) in tensors.
tf.string 是一种 dtype，也就是说，在张量中，您可以用字符串 (可变长度字节数组) 来表示数据。

The strings are atomic and cannot be indexed the way Python strings are. The length of the string is not one of the axes of the tensor. See tf.strings for functions to manipulate them.
字符串是原子类型，无法像 Python 字符串一样编制索引。字符串的长度并不是张量的一个轴。有关操作字符串的函数，请参阅 tf.strings。

Here is a scalar string tensor:
下面是一个标量字符串张量：

# Tensors can be strings, too here is a scalar string.
scalar_string_tensor = tf.constant("Gray wolf")
print(scalar_string_tensor)tf.Tensor(b'Gray wolf', shape=(), dtype=string)

And a vector of strings:
下面是一个字符串向量：

# If you have three string tensors of different lengths, this is OK.
tensor_of_strings = tf.constant(["Gray wolf","Quick brown fox","Lazy dog"])
# Note that the shape is (3,). The string length is not included.
print(tensor_of_strings)tf.Tensor([b'Gray wolf' b'Quick brown fox' b'Lazy dog'], shape=(3,), dtype=string)

In the above printout the b prefix indicates that tf.string dtype is not a unicode string, but a byte-string. See the Unicode Tutorial for more about working with unicode text in TensorFlow.
https://www.tensorflow.org/text/guide/unicode
在上面的打印输出中，b 前缀表示 tf.string dtype 不是 unicode 字符串，而是字节字符串。

If you pass unicode characters they are utf-8 encoded.
如果传递 unicode 字符，则会使用 utf-8 编码。

tf.constant("												


											
Tensor (张量) - 神经网络中的数据结构相关推荐
	

								
【TensorFlow】——不同shape的tensor在神经网络中的应用（scalar，vector，matrix）
		
目录 ​ 1.scalar--标量 1)在神经网络中存在的场景 2)one_hot编码 3)举例应用 2.vector--向量 ​ 3.matrixs--矩阵 4.dim=3的tensor 5.dim ...
		
						
DeepMind发布《神经网络中持续学习》Cell综述论文
		
点上方计算机视觉联盟获取更多干货 仅作学术分享,不代表本公众号立场,侵权联系删除 转载于:专知 AI博士笔记系列推荐 周志华<机器学习>手推笔记正式开源!可打印版本附pdf下载链接 现代机 ...
		
						
（tensorflow笔记）神经网络中的一些关键概念（学习率、激活函数、损失函数、欠拟合和过拟合、正则化和优化器）
		
目录 1.神经网络复杂度 空间复杂度 时间复杂度 2.学习率策略 指数衰减学习率 分段常数衰减 3.激活函数 sigmoid tanh ReLU Leaky ReLU 建议 4.损失函数 均方误差损失 ...
		
						
在张量运算中使用“Torch”
		
讨论5种基本和最常用的张量运算 深度学习使我们能够执行非常复杂的任务.为了有效地执行任务,我们需要一个灵活的工具.由于其简单性,Pytorch为我们提供了此选项.它使用GPU(图形处理单元)提供加速的 ...
		
						
神经网络中的降维和升维方法 (tensorflow  pytorch)
		
大名鼎鼎的UNet和我们经常看到的编解码器模型,他们的模型都是先将数据下采样,也称为特征提取,然后再将下采样后的特征恢复回原来的维度.这个特征提取的过程我们称为"下采样",这个恢复 ...
		
						
高效Tensor张量生成
		
高效Tensor张量生成 Efficient Tensor Creation 从C++中的Excel数据中创建Tensor张量的方法有很多种,在简单性和性能之间都有不同的折衷.本文讨论了一些方法及其权 ...
		
						
神经网络中的权重初始化一览：从基础到Kaiming
		
点击上方"小白学视觉",选择加"星标"或"置顶" 重磅干货,第一时间送达 在进行各种小实验和思维训练时,你会逐步发现为什么在训练深度神经网络 ...
		
						
神经网络中的注意力机制总结及PyTorch实战
		
技术交流 QQ 群:1027579432,欢迎你的加入! 欢迎关注我的微信公众号:CurryCoder的程序人生 0.概述 当神经网络来处理大量的输入信息时,也可以借助人脑的注意力机制,只选择一些关键 ...
		
						
神经网络中的「注意力」是什么？怎么用？
		
来源:转载自公众号「雷克世界」微信号:ROBO_AI 编译:嗯~阿童木呀.多啦A亮 概要:神经网络中的注意力机制(Attention mechanisms),也被称为"神经注意力" ...
		
		

					
最新文章
	

						
虚拟机上SourceInsight访问Linux系统的代码
		
						
springboot实战 获取spring上下文的4种方式
		
						
命名分组(?name....)
		
						
精益分析体系构建如何“因企制宜”？
		
						
实战分享｜数据驱动「付费转化」的3个思路
		
						
关于创建SWAP示例
		
						
微信公众平台客户端调试工具
		
						
【渝粤教育】国家开放大学2018年秋季 0550-22T素描（一） 参考试题
		
						
上传附件点击事件_支持高拍仪扫描凭证附件的出纳记账软件
		
						
String a = new String(“abc“)； 到底创建了几个对象
		
						
如何卸载office201032位_如何卸载流氓版office2010
		
						
康师傅就“水源事件”发声明 称矿物质水合标
		
						
HBO宣布续订《西部世界》第三季
		
						
路由器更换wan口及vlan配置
		
						
Python爬虫 之 异步爬虫
		
						
将阿拉伯数字转换成罗马数字 Integer to Roman
		
						
TcaplusDB亮相全球云计算大会 一举斩获优秀解决方案奖
		
						
分享一个开源的QT的串口示波器
		
						
SafeSignCertReg.exe导致系统工作异常
		
						
图形化开放式生信分析系统开发- 1基本需求分析及技术实现
		
		
	

热门文章
	

									
利用python进行保险数据分析及可视化
			
						
http服务器的实现
			
						
克鲁斯卡尔算法与普里姆算法详解
			
						
自学前端需要达到什么水平才能去找工作？来看看这套前端学习路线图
			
						
python刷直播人气_python3爬取斗鱼某些版块的主播人气
			
						
打破 FOXMAIL 疯狂占用磁盘读写资源的魔障
			
						
2019微软Power BI 每月功能更新系列——2月Power BI 新功能学习
			
						
中国计算机科学家陈什元,我国著名计算机科学家、西安交通大学郑守淇教授逝世...
			
						
WinDbg+WMware+Vista
			
						
linux shell脚本