torch

Appearance

torch.Tensor ​

Tensor ​

A torch.Tensor is a multi-dimensional matrix containing numbers.

Initializing Tensors ​

A tensor can be constructed from a Python list or sequence using the torch.tensor() constructor:

python
>>> torch.tensor([[1., -1.], [1., -1.]])
tensor([[1, -1], [1, -1]])
>>> torch.tensor([1, 2, 3, 4])
tensor([1, 2, 3, 4])

You can also use tensor creation ops:

python
>>> torch.zeros(2, 3)
tensor([[0, 0, 0], [0, 0, 0]])
>>> torch.ones(2, 3)
tensor([[1, 1, 1], [1, 1, 1]])
>>> torch.randn(2, 3)
tensor([[0.04188625638329904, -0.2835499126703179, 0.15911637825403122], [-0.35108857534104865, 1.606089739430864, 0.013343448166239665]])

Tensor Attributes ​

AttributeDescription
shapeReturns the shape of the tensor.
dataReturns a detached view of the tensor data (no gradient).
requires_gradIs True if gradients need to be computed for this Tensor.
gradThis attribute is None by default and becomes a Tensor the first time a call to backward() computes gradients for self.
TReturns a view of this tensor with its dimensions reversed.

Tensor Methods ​

The contents of a tensor can be accessed using Python’s indexing and slicing notation:

python
>>> x = torch.tensor([[1, 2, 3], [4, 5, 6]])
>>> x[0]
tensor([1, 2, 3])
>>> x[1, 2]
tensor(6)
>>> x[0:2]
tensor([[1, 2, 3], [4, 5, 6]])

Integer indexing selects along dimension 0 and returns a tensor with one fewer dimension. Tuple indexing chains dimension-0 selections (e.g. x[i, j] is equivalent to x[i][j]). Slice indexing returns a new tensor containing the selected rows.

Use torch.Tensor.item() to get a Python number from a tensor containing a single value:

python
>>> x = torch.tensor([[1]])
>>> x
tensor([[1]])
>>> x.item()
1
>>> x = torch.tensor(2.5)
>>> x
tensor(2.5)
>>> x.item()
2.5
MethodDescription
tolist()Return tensor data as a (nested) Python list, or a Python scalar for 0-d tensors.
item()Returns the value of this tensor as a standard Python number.
size(dim=None)Returns the size of the self tensor.
dim()Returns the number of dimensions of self tensor.
numel()Returns the total number of elements in the self tensor.
backward(gradient=None)Computes the gradient of current tensor w.r.t. graph leaves.
detach()Returns a new Tensor, detached from the current graph.
zero_()Fills self tensor with zeros.
retain_grad()Enables .grad attribute for non-leaf Tensors.
reshape(*args)Returns a tensor with the same data and number of elements as self but with the specified shape.
view(*args)Returns a new tensor with the same data as the self tensor but of a different shape.
squeeze(dim=None)Returns a tensor with all specified dimensions of input of size 1 removed.
unsqueeze(dim)Returns a new tensor with a dimension of size one inserted at the specified position.
expand(*args)Returns a new view of the self tensor with singleton dimensions expanded to a larger size.
transpose(dim0, dim1)Returns a tensor that is a transposed version of self.
flatten(start_dim=0, end_dim=-1)Flattens self tensor by reshaping it into a one-dimensional tensor.

Reductions ​

MethodDescription
sum(dim=None, keepdim=False)Returns the sum of all elements in the tensor.
mean(dim=None, keepdim=False)Returns the mean value of all elements in the tensor.
max(dim=None, keepdim=False)Returns the maximum value of all elements in the tensor.
min(dim=None, keepdim=False)Returns the minimum value of all elements in the tensor.

Math Operations ​

Many math operations are exposed directly as tensor methods: add(other), sub(other), mul(other), div(other), pow(other), matmul(other), neg(), abs(), log(), exp(), sqrt(), square(), sin(), cos(), tan(), sigmoid(), relu(), sign(), reciprocal(), nan_to_num().

Additionally, the Python __add__, __sub__, __mul__, __truediv__, __pow__, and __matmul__ magic methods are implemented to match standard math operators (+, -, *, /, **, @).

Comparison Operations ​

MethodDescription
lt(other)Computes self < other element-wise.
gt(other)Computes self > other element-wise.
le(other)Computes self <= other element-wise.
ge(other)Computes self >= other element-wise.
eq(other)Computes self == other element-wise.
ne(other)Computes self != other element-wise.
allclose(other, rtol=1e-5, atol=1e-8, equal_nan=False)This function checks if self and other satisfy the condition.

Method and Attribute Details ​

torch.Tensor.shape PyTorch↗ ​

Returns the shape of the tensor as a tuple.

python
>>> x = torch.tensor([[1, 2], [3, 4]])
>>> x.shape
(2, 2)

torch.Tensor.data ​

Returns a detached view of the tensor data (no gradient).

python
>>> x = torch.tensor([1., 2.], requires_grad=True)
>>> x.data
tensor([1, 2])
>>> x.data.requires_grad
False

torch.Tensor.requires_grad PyTorch↗ ​

Is True if gradients need to be computed for this Tensor, False otherwise.

python
>>> x = torch.tensor([1., 2.], requires_grad=True)
>>> x.requires_grad
True

torch.Tensor.grad PyTorch↗ ​

This attribute is None by default and becomes a Tensor the first time a call to backward() computes gradients for self.

python
>>> x = torch.tensor([1., 2.], requires_grad=True)
>>> y = x.sum()
>>> y.backward()
>>> x.grad
tensor([1, 1])

torch.Tensor.T ​

Returns a view of this tensor with its dimensions reversed.

python
>>> x = torch.tensor([[1, 2], [3, 4]])
>>> x.T
tensor([[1, 3], [2, 4]])

WARNING

The use of Tensor.T on tensors of dimension other than 2 to reverse their shape is deprecated and it will throw an error.

torch.Tensor.tolist PyTorch↗ ​

python
Tensor.tolist() -> list

Return tensor data as a (nested) Python list, or a Python scalar for 0-d tensors.

python
>>> x = torch.tensor([[1, 2], [3, 4]])
>>> x.tolist()
[[1, 2], [3, 4]]

torch.Tensor.item PyTorch↗ ​

python
Tensor.item() -> number

Returns the value of this tensor as a standard Python number. This only works for tensors with one element.

python
>>> x = torch.tensor([3])
>>> x.item()
3

torch.Tensor.size PyTorch↗ ​

python
Tensor.size(dim=None) -> tuple or int

Returns the size of the self tensor. If dim is not specified, returns the full shape tuple.

python
>>> x = torch.tensor([[1, 2], [3, 4]])
>>> x.size()
(2, 2)
>>> x.size(1)
2

torch.Tensor.dim PyTorch↗ ​

python
Tensor.dim() -> int

Returns the number of dimensions of self tensor.

python
>>> x = torch.tensor([[1, 2], [3, 4]])
>>> x.dim()
2

torch.Tensor.numel PyTorch↗ ​

python
Tensor.numel() -> int

Returns the total number of elements in the self tensor.

python
>>> x = torch.tensor([[1, 2], [3, 4]])
>>> x.numel()
4

torch.Tensor.backward PyTorch↗ ​

python
Tensor.backward(gradient=None)

Computes the gradient of current tensor w.r.t. graph leaves.

python
>>> x = torch.tensor([1., 2.], requires_grad=True)
>>> y = x.sum()
>>> y.backward()
>>> x.grad
tensor([1, 1])

torch.Tensor.detach PyTorch↗ ​

python
Tensor.detach() -> Tensor

Returns a new Tensor, detached from the current graph.

python
>>> x = torch.tensor([1., 2.], requires_grad=True)
>>> y = x.detach()
>>> y.requires_grad
False

torch.Tensor.zero_ PyTorch↗ ​

python
Tensor.zero_() -> Tensor

Fills self tensor with zeros.

python
>>> x = torch.tensor([1., 2.])
>>> x.zero_()
tensor([0, 0])
>>> x
tensor([0, 0])

torch.Tensor.retain_grad PyTorch↗ ​

python
Tensor.retain_grad()

Enables .grad attribute for non-leaf Tensors.

python
>>> x = torch.tensor([1., 2.], requires_grad=True)
>>> y = x * 2
>>> y.retain_grad()
>>> z = y.sum()
>>> z.backward()
>>> y.grad
tensor([1, 1])

torch.Tensor.reshape PyTorch↗ ​

python
Tensor.reshape(*args) -> Tensor

See torch.reshape.

torch.Tensor.view PyTorch↗ ​

python
Tensor.view(*args) -> Tensor

Returns a new tensor with the same data as the self tensor but of a different shape.

python
>>> x = torch.tensor([1, 2, 3, 4])
>>> x.view(2, 2)
tensor([[1, 2], [3, 4]])

torch.Tensor.squeeze PyTorch↗ ​

python
Tensor.squeeze(dim=None) -> Tensor

See torch.squeeze.

torch.Tensor.unsqueeze PyTorch↗ ​

python
Tensor.unsqueeze(dim) -> Tensor

See torch.unsqueeze.

torch.Tensor.expand PyTorch↗ ​

python
Tensor.expand(*args) -> Tensor

Returns a new view of the self tensor with singleton dimensions expanded to a larger size.

python
>>> x = torch.tensor([1])
>>> x.expand(3)
tensor([1, 1, 1])

torch.Tensor.transpose PyTorch↗ ​

python
Tensor.transpose(dim0, dim1) -> Tensor

See torch.transpose.

torch.Tensor.flatten PyTorch↗ ​

python
Tensor.flatten(start_dim=0, end_dim=-1) -> Tensor

See torch.flatten.

torch.Tensor.sum PyTorch↗ ​

python
Tensor.sum(dim=None, keepdim=False) -> Tensor

See torch.sum.

torch.Tensor.mean PyTorch↗ ​

python
Tensor.mean(dim=None, keepdim=False) -> Tensor

See torch.mean.

torch.Tensor.max PyTorch↗ ​

python
Tensor.max(dim=None, keepdim=False) -> Tensor

See torch.max.

torch.Tensor.min PyTorch↗ ​

python
Tensor.min(dim=None, keepdim=False) -> Tensor

See torch.min.

torch.Tensor.add PyTorch↗ ​

python
Tensor.add(other) -> Tensor

See torch.add.

torch.Tensor.sub PyTorch↗ ​

python
Tensor.sub(other) -> Tensor

See torch.sub.

torch.Tensor.mul PyTorch↗ ​

python
Tensor.mul(other) -> Tensor

See torch.mul.

torch.Tensor.div PyTorch↗ ​

python
Tensor.div(other) -> Tensor

See torch.div.

torch.Tensor.pow PyTorch↗ ​

python
Tensor.pow(other) -> Tensor

See torch.pow.

torch.Tensor.matmul PyTorch↗ ​

python
Tensor.matmul(other) -> Tensor

See torch.matmul.

torch.Tensor.neg PyTorch↗ ​

python
Tensor.neg() -> Tensor

See torch.neg.

torch.Tensor.abs PyTorch↗ ​

python
Tensor.abs() -> Tensor

See torch.abs.

torch.Tensor.log PyTorch↗ ​

python
Tensor.log() -> Tensor

See torch.log.

torch.Tensor.exp PyTorch↗ ​

python
Tensor.exp() -> Tensor

See torch.exp.

torch.Tensor.sqrt PyTorch↗ ​

python
Tensor.sqrt() -> Tensor

See torch.sqrt.

torch.Tensor.square PyTorch↗ ​

python
Tensor.square() -> Tensor

See torch.square.

torch.Tensor.sin PyTorch↗ ​

python
Tensor.sin() -> Tensor

See torch.sin.

torch.Tensor.cos PyTorch↗ ​

python
Tensor.cos() -> Tensor

See torch.cos.

torch.Tensor.tan PyTorch↗ ​

python
Tensor.tan() -> Tensor

See torch.tan.

torch.Tensor.sigmoid PyTorch↗ ​

python
Tensor.sigmoid() -> Tensor

See torch.sigmoid.

torch.Tensor.relu ​

python
Tensor.relu() -> Tensor

See torch.relu.

torch.Tensor.sign PyTorch↗ ​

python
Tensor.sign() -> Tensor

See torch.sign.

torch.Tensor.reciprocal PyTorch↗ ​

python
Tensor.reciprocal() -> Tensor

See torch.reciprocal.

torch.Tensor.nan_to_num PyTorch↗ ​

python
Tensor.nan_to_num() -> Tensor

See torch.nan_to_num.

torch.Tensor.lt PyTorch↗ ​

python
Tensor.lt(other) -> Tensor

See torch.lt.

torch.Tensor.gt PyTorch↗ ​

python
Tensor.gt(other) -> Tensor

See torch.gt.

torch.Tensor.le PyTorch↗ ​

python
Tensor.le(other) -> Tensor

See torch.le.

torch.Tensor.ge PyTorch↗ ​

python
Tensor.ge(other) -> Tensor

See torch.ge.

torch.Tensor.eq PyTorch↗ ​

python
Tensor.eq(other) -> Tensor

See torch.eq.

torch.Tensor.ne PyTorch↗ ​

python
Tensor.ne(other) -> Tensor

See torch.ne.

torch.Tensor.allclose PyTorch↗ ​

python
Tensor.allclose(other, rtol=1e-5, atol=1e-8, equal_nan=False) -> bool

See torch.allclose.