How to code neural networks (Part 2)

 

      Photo by Mohammad Rahmani on Unsplash

Welcome to part 2 of this article. In the previous article, I discussed the theory and working of a perceptron. If you haven't read that article I strongly suggest you read it before continuing with this one. Click this link to go there

In this article, we are going to start implementing all we learned in code. So let's dive in.

We will start off with the activation function. We have discussed the function we are going to be using in the previous article.

function activation(x){
    if(x <= 0.5){
        return 0;
    }else{
        return 1;
    }
}

This is just a basic function that returns 0 if the value is less than or equal to 0.5 and 1 otherwise.

We are going to make a  perceptron class that can be used to solve many different problems. For my example, I will be continuing with the AND gate example. 

We can start the class with a basic constructor. We know that a perceptron must have a weights array, number of inputs, and learning rate. So our class looks something like this: 

class Perceptron{
    constructor(input_nodes){
        this.input_nodes = input_nodes; //number of inputs
        this.weights = []; //array to store all weights for all inputs
        this.lr = 0.1; //Learning rate
    }
}

Now we will move to initialize the weights, for this example, I will just start off with some random numbers and let the perceptron figure out the right ones on its own. Let's add that to the constructor.

class Perceptron{
    constructor(input_nodes){
        this.input_nodes = input_nodes;
        this.weights = [];
        this.lr = 0.1;

        for(let i = 0; i < this.input_nodes; i++){
            this.weights[i] = Math.random();
        }
    }
}

Here I am just using a for-loop to assign random numbers to the weights. Note that the number of weights initialized is equal to the number of inputs the perceptron will receive.

If you understood everything till now, Good job! Now we will start with getting an output from the perceptron. Lets name the function guess(). This function will receive an array whose length is equal to the number of input nodes. It will calculate the weighted sum and pass that through the activation function. This function looks like this: 

guess(input){
        let weighted_sum = 0;
        for(let i = 0; i < this.input_nodes; i++){
            weighted_sum += input[i] * this.weights[i];
        }
        let output = activation(weighted_sum)
        console.log("Output: "+output);
        return output;
    }

Don't worry, I will break this down for you. First I initialized a variable to store the weighted sum. Then I looped through and calculated the product of input with its respective weight. I kept on adding this value to the weighted sum for all the inputs.

Once we have to weighted sum, we can pass that through the activation function we made earlier. That it! we can have an output from the perceptron now. Let's try that once. To get an output first initialize an object from our class and run the guess() method with some inputs.

let p1 = new Perceptron(2); // the 2 means it take two inputs
p1.guess([0,1]);  //notice array of length 2 passed as argument

Give it a shot, you should get something like this in the console. 

If you got an output that looks like this, Congrats. You have made a perceptron from scratch. If you don't get output or have any errors, feel free to ask In the comment section. I will help you out.

Now we will start off with the most exciting part, actually training our perceptron. I will be training our perceptron to learn AND gate. We will be using supervised learning to train our perceptron. I have already discussed that training means adjusting the weights of our perceptron. Let's make a function train() that takes two parameters: array of the sample inputs and correct output. 

train(input, correct_output){
        let perceptron_output = this.guess(input);
        let error = correct_output - perceptron_output;
    }

This function gets the output from our perceptron and compares it with the correct output provided by us. The difference in these values is the error. We will use this error to adjust our weights accordingly

Now we need to loop through the weights and calculate the delta weights. This is the change in weights to be made to make the outputs more accurate. I had explained the formula and the working in the previous article. Let's add that part to the function: 

train(input, correct_output){
        let perceptron_output = this.guess(input);
        let error = correct_output - perceptron_output;
        for(let i = 0; i < this.input_nodes; i++){
            let delta_weight = this.weights[i] * error * this.lr;
            this.weights[i] += delta_weight;
        }
    }

And that's it. All we need to do now is run that training function many times and over time the perceptron will give more and more accurate answers. Make sure you train the perceptron with different inputs. To do this I will just call this method in a for loop: 

for(let i = 0; i < 1000; i++){
    p1.train([0,0], 0);
    p1.train([0,1], 0);
    p1.train([1,0], 0);
    p1.train([1,1], 1);
}

Congratulations! You have successfully taught your perceptron to recognize AND gate. As a fun exercise, make another object from the main class and train it to recognize OR gate. 

Our model is really simple, so it cannot understand more complex problems. You can use this knowledge to understand more complex networks and how to train them.

~Arvind Suthar