Build a Neural Network With Pytorch

PyTorch basics

In the previous lessons, we looked at some Pytorch code, but this time we will take a closer look at it.

Pytorch is an open-source Python deep learning framework that enables us to build and train neural networks. Pytorch is a library that helps you perform mathematical operations between matrices in their basic form because, as you will realize, deep learning is just simple linear algebra.

The fundamental building block of a Pytorch is the tensor. A tensor is an N-dimensional array. We can have an 1d array (or a vector) x=[1,2,3,4,5], a 2d-array y=[[1,2],[3,4]], and so on.

In Pytorch, these can be defined as:

X= torch.tensor([ 1,2,3,4,5]) 
Y= torch.tensor([1,2],[3,4]]) 

From there we can define almost all mathematical operations between tensors.

Z = torch.add(X,Y) 
Z = torch.matmul(X,Y) 
Z = 1 / (1+torch.exp(x)) 

Let’s revisit the neuron’s equation: $a =f(w_1*a_1 + w_2*a_2 + w_3*a_3 + b_o)$

The above equation can be easily transformed into tensor operations (remember everything in deep learning can be represented as tensors).

[a_4] = f([ w1, w2, w3] * [a1, a2,a3 ] +[b_o])

All we did here is this:

• Gather together all activations into a 1-d vector and all weights into another 1-d vector.
• Multiply them.
• Add the bias.
• Apply the sigmoid function in the result.

Note that individual numbers can also be seen as 0d tensors.

Let’s proceed with our first exercise. Using everything that you learned just now, try to program your first neuron from scratch. All necessary information and commands have already been mentioned, so all you have to do is reconstruct the above equation using Pytorch.

As a first exercise, try to code a simple neuron with three inputs in Pytorch. Initialize the weights as [0.5,0.5,0.5], the bias as 0.5, and return the output.