PyTorch is a popular deep learning framework that provides a wide range of functionalities for working with tensors, which are multi-dimensional arrays. In this article, we will explore various tensor operations in PyTorch, focusing on addition, multiplication, division, and matrix multiplication.
Bare in mind that this post is a continuation of my previous post Introduction to PyTorch and I may take for granted some concepts like how to import libraries etc.
PyTorch offers different syntaxes for performing tensor operations. Let's start by examining the addition operation and its syntax variations.
Remember from last post the tensor x was:
tensor([[0.9476, 0.8858, 0.3435],
[2.0000, 2.0000, 2.0000],
[2.0000, 2.0000, 2.0000],
[0.3921, 0.2402, 0.8420],
[0.8632, 0.6434, 0.9973]])
The first syntax for addition involves using the + operator.
y = torch.rand(5, 3)
print(x + y)
Output:
tensor([[1.9114, 0.9414, 1.0422],
[2.4761, 2.0043, 2.3640],
[2.3663, 2.0935, 2.1085],
[0.5095, 0.3583, 0.8873],
[1.6496, 0.9493, 1.8484]])
The code above creates a random tensor y of size (5, 3) and performs element-wise addition between x and y. The result is printed, showing the addition of corresponding elements in the tensors.
The second syntax for addition is using the torch.add() function.
print(torch.add(x, y))
In this case, the torch.add() function is used to add x and y tensors element-wise. The result is the same as the previous syntax.
Outout:
tensor([[1.9114, 0.9414, 1.0422],
[2.4761, 2.0043, 2.3640],
[2.3663, 2.0935, 2.1085],
[0.5095, 0.3583, 0.8873],
[1.6496, 0.9493, 1.8484]])
The third syntax allows specifying an output tensor for storing the result of the addition operation.
result = torch.Tensor(5, 3)
torch.add(x, y, out=result)
print(result)
Output:
tensor([[1.9114, 0.9414, 1.0422],
[2.4761, 2.0043, 2.3640],
[2.3663, 2.0935, 2.1085],
[0.5095, 0.3583, 0.8873],
[1.6496, 0.9493, 1.8484]])
Here, a tensor result of size (5, 3) is created, and the torch.add() function is used with the out argument to store the result in the result tensor.
PyTorch also provides in-place operations, denoted by the _ suffix. In-place operations modify the tensor itself.
y.add_(x)
print(y)
Output:
tensor([[1.9114, 0.9414, 1.0422],
[2.4761, 2.0043, 2.3640],
[2.3663, 2.0935, 2.1085],
[0.5095, 0.3583, 0.8873],
[1.6496, 0.9493, 1.8484]])
In this example, y tensor is modified in-place by adding x to it. The result is printed, showing the updated y tensor.
Bare in mind that any operation that mutates a tensor in-place is post-fixed with an _. For example: x.copy_(y), x.t_(), will change x.
The next set of operations we'll explore are multiplication and division of tensors.
x = torch.zeros([2, 2]) + 5
x[1, :] = 10
y = torch.zeros([2, 2]) + 2
print(x, y)
Output:
tensor([[ 5., 5.],
[10., 10.]]),
tensor([[2., 2.],
[2., 2.]])
We can simply use the * operator
print(x * y)
Output:
tensor([[10., 10.],
[20., 20.]])
The code above demonstrates multiplication of two tensors, x and y, using the * operator. Element-wise multiplication is performed between corresponding elements of the tensors.
Alternatively, the torch.multiply() function can be used:
torch.multiply(x, y)
Both approaches yield the same result.
In-place multiplication modifies the tensor itself. Here's an example:
x.multiply_(x)
print(x)
The x tensor is multiplied by itself in-place, resulting in the modified tensor.
Output:
tensor([[25., 25.],
[100., 100.]])
The division operator / can be used to perform element-wise division between tensors. In this case, x is divided by y element-wise.
print(x / y)
Output:
tensor([[2.5000, 2.5000],
[5.0000, 5.0000]])
torch.div(x,y)
Both approaches produce the same result.
In-place division modifies the tensor itself. Here's an example:
x.div_(y)
The x tensor is divided by y in-place, resulting in the modified tensor x with same values as the cells above.
PyTorch provides the torch.matmul() function for matrix multiplication.
x = torch.zeros([2,4]) + 5
x[1,:] = 10
y = torch.zeros([4,2]) + 2
print(x, y)
Output:
tensor([[ 5., 5., 5., 5.],
[10., 10., 10., 10.]]),
tensor([[2., 2.],
[2., 2.],
[2., 2.],
[2., 2.]])
torch.matmul(x,y)
tensor([[40., 40.],
[80., 80.]])
In the code above, we perform matrix multiplication between tensors x and y using the torch.matmul() function. The tensor xhas a size of (2, 4) and y has a size of (4, 2). The result of the matrix multiplication is computed and returned as a new tensor. Matrix multiplication involves multiplying corresponding elements of rows in the first tensor with columns in the second tensor and summing them up to obtain the resulting elements of the output tensor. The resulting tensor from the matrix multiplication has a size of (2, 2) because the number of columns in the first tensor matches the number of rows in the second tensor. Overall, PyTorch provides a comprehensive set of tensor operations, including addition, multiplication, division, and matrix multiplication. These operations can be performed using various syntaxes, allowing flexibility and ease of use. Understanding these operations is crucial for working with tensors in PyTorch and implementing deep learning algorithms effectively.