Tensor in PyTorch

In PyTorch, a tensor is a multi-dimensional array with each element, by default, stored as a 32-bit floating point value.

When compared to the list of lists found in core Python, a tensor in PyTorch is a multi-dimensional array that has two key distinctions.
1. The difference is that standard Python lists are handled as individual objects while tensor(s) are allotted memory in a contiguous block.

The figure above depicts the structure of the python list and PyTorch tensor in memory 

2. While the Python list program is only compatible with central processing units (CPUs), the tensor is also compatible with graphics processing units (GPUs)
NumPy array elements are likewise stored on contiguous memory blocks, but NumPy arrays can only run on CPUs as of now. Tensors are optimized for mathematical operations specifically when dealing with a large amount of data. 

Installing PyTorch

For the code samples below to work on your computer, you need to have the PyTorch library installed or you can use free Google Colab Notebooks to run code. Colab comes with pre-installed PyTorch and a free GPU.

Installing using pip

pip install torch

on mac

pip3 install torch

using conda

conda install pytorch -c pytorch

Creating a PyTorch Tensor

import torch
x = torch.tensor([0.4, 1.23, 1.0, 0.3, 5.5])
print(x)
# output: tensor([0.4000, 1.2300, 1.0000, 0.3000, 5.5000])

Moving tensor on GPU 

#checking if cuda is available 
torch.cuda.is_available()

#number of GPUs available
print(torch.cuda.device_count())

#set device gpu if available
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
x = x.to(device=device)
print(x)

#OUTPUT on CPU
#False
#0
#tensor([0.4000, 1.2300, 1.0000, 0.3000, 5.5000])

#OUTPUT after enabling GPU on colab
#True
#1
#tensor([0.4000, 1.2300, 1.0000, 0.3000, 5.5000], device='cuda:0')

to enable GPU on google colab on top navigation select Runtime>Change runtime type>Hardware accelerator>GPU>save and run all cells again.

Numeric Types in PyTorch

A tensor's datatype is used to define the format and kind of data that will be included in it. To indicate what sort of numerical data will be included in the tensor, the dtype parameter is used when building tensors. It is possible to learn more about the type, range, precision, and amount of bytes per value by using a specific numeric type, such as int8, int16, float32, folat64, uint8, bool, and so on. By default, PyTorch holds the tensors with 32-bit floating point precision.

To get dtype of a tensor 

print(x.dtype)
#torch.float32