# The inevitable assignment in your Deep learning class: Neural networks using only numpy

## Comes in handy during interviews. For beginners and experts alike

## Defining a Neural network

The way we define the architecture of a fully connected feed-forward neural network is using hyperparameters, mentioning the number of layers, the number of neurons in each layer and the activation function for each layer.

The reason the output of the previous layer and input of the next layer is repeated is for ease of parsing it and is also intuitive to look at.

## The basics

The neural network depending on its architecture will have weights, biases and activation functions. We also need the network to do forward propagation (compute y given x), backward propagation (update weights given the ground truth and prediction). For simplicity, we only train the network with MSE (Mean squared error) loss and SGD (Stochastic Gradient Descent).

So we will start with a basic class definition that provides the above features.

## Building the network

Now that we know the hyperparameters we can initialise weight matrices and store them in member variables. `weights[i]`

is a numpy array containing the weight matrix of layer i. The same holds for `biases[i]`

and `activation_functions[i]`

.

## Activation functions

Keeping the network aside, for now, we will ready up the activation functions. We use 2 simple commonly used activation functions sigmoid and relu. You are welcome to use your own. It is just plug-and-play cause of the way the rest of the code is written.

The derivative definition of each activation function is essential to implement the backpropagation part.

We now define a dictionary to refer to function definition from strings. If you are using other activation functions just make sure you include them in these dictionaries and the rest of the code will work fine.

**Forward propagation**

We now have all the necessary elements to write the `forward()`

function. Given an input vector, the function needs to sequentially multiply it with the weight matrix, add bias and apply activation function for each layer.

I hope you followed along and were able to write functioning code till now. I believe backpropagation is a bit involved so I’ve written a separate article to explain that thoroughly.

Thanks for reading. Do comment your queries and suggestions.