Ctrl+K

Basic Operations with Tensors#

Introduction#

Tensors are the fundamental data structures in PyTorch, representing multi-dimensional arrays. Understanding basic tensor operations is crucial for effectively working with PyTorch and building neural networks. In this tutorial, we’ll cover the essential operations you need to know when working with tensors in PyTorch.

Prerequisites#

  • Basic understanding of Python programming.

  • Familiarity with NumPy arrays would be helpful but not required.

Outline#

  1. Creating Tensors

    • Creating tensors from data.

    • Creating tensors with specific sizes and data types.

    • Generating tensors with random values.

  2. Indexing and Slicing

    • Accessing individual elements.

    • Slicing tensors to extract sub-tensors.

  3. Tensor Operations

    • Element-wise operations (addition, subtraction, multiplication, division).

    • Reduction operations (sum, mean).

    • Matrix operations (matrix multiplication, transposition).

  4. Broadcasting

    • Understanding broadcasting rules.

    • Performing operations on tensors with different shapes.

  5. In-place Operations

    • Modifying tensors in-place for efficiency.

  6. Reshaping and Resizing

    • Changing the shape and size of tensors.

  7. Device Management

    • Moving tensors between CPU and GPU devices.

  8. Data Type Conversion

    • Converting tensors to different data types.

  9. Error Handling

    • Handling common errors when working with tensors.

Detailed Tutorial#

1. Creating Tensors#

Creating Tensors from Data#

import torch

# From Python list
tensor_list = torch.tensor([1, 2, 3, 4, 5])

# From NumPy array
import numpy as np
numpy_array = np.array([[1, 2, 3], [4, 5, 6]])
tensor_numpy = torch.tensor(numpy_array)

# Random tensor
tensor_random = torch.rand(3, 3)

Creating Tensors with Specific Sizes and Data Types#

# Zeros tensor
tensor_zeros = torch.zeros(2, 3)

# Ones tensor
tensor_ones = torch.ones(3, 2)

# Tensor with specific data type
tensor_float = torch.ones(2, 2, dtype=torch.float32)
tensor_long = torch.zeros(3, 3, dtype=torch.int64)

Generating Tensors with Random Values#

# Random tensor (uniform distribution)
tensor_uniform = torch.rand(3, 3)

# Random tensor (normal distribution)
tensor_normal = torch.randn(2, 2)

2. Indexing and Slicing#

# Accessing individual elements
print(tensor_random[0, 0])

# Slicing tensors
print(tensor_random[:, 1:])

3. Tensor Operations#

Element-wise Operations#

# Addition
result_add = tensor_float + tensor_ones

# Subtraction
result_sub = tensor_float - tensor_ones

# Multiplication
result_mul = tensor_float * tensor_ones

# Division
result_div = tensor_float / tensor_ones

Reduction Operations#

# Sum
result_sum = torch.sum(tensor_random)

# Mean
result_mean = torch.mean(tensor_random)

Matrix Operations#

# Matrix multiplication
result_matmul = torch.matmul(tensor_float, tensor_ones)

# Transpose
result_transpose = torch.transpose(tensor_random, 0, 1)

4. Broadcasting#

# Broadcasting a scalar
scalar = 2
result_broadcast = tensor_float + scalar

5. In-place Operations#

# In-place addition
tensor_float.add_(5)

6. Reshaping and Resizing#

# Reshape tensor
tensor_reshaped = tensor_random.view(3, -1)

# Resize tensor (in-place)
tensor_random.resize_(2, 6)

7. Device Management#

# Move tensor to GPU if available
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
tensor_gpu = tensor_random.to(device)

8. Data Type Conversion#

# Convert tensor to different data type
tensor_float32 = tensor_long.to(torch.float32)

9. Error Handling#

# Example of a common error
tensor_a = torch.tensor([1, 2, 3])
tensor_b = torch.tensor([1, 2, 3, 4])
result = tensor_a + tensor_b  # This will raise a RuntimeError due to incompatible shapes

Conclusion#

This tutorial covered the basic operations you need to know when working with tensors in PyTorch. By mastering these operations, you’ll be well-equipped to manipulate tensors effectively and build complex neural networks. Experiment with different operations and explore their applications to deepen your understanding further.

On this page