Introduction to Machine Learning in Plain JavaScript
Build a simple Perceptron algorithm with two inputs and a single output
In this example, we will think of a simplified case (image above) where we have a plant that is monitored by two sensors one will monitor if the plant gets enough sun and the other will monitor if it gets enough water.
Both sensors will just send 0 or 1, 0 stands for the resource is missing and 1 for perfect feeding the plant with the resource.
We want to build a perceptron (simple machine learning algorithm) to calculate if our plant is getting enough water and sun if something is missing it should give output 1 which means our little robot needs to take action and take care of our plant.
In the first part, we feed our algorithm with inputs.
These pairs would be an array of two values [0, 0], [0, 1], [1, 0] and [1, 1].
In machine learning, we feed our algorithm with data (normally it’s a ton of them) and our algorithm learns from them and sets up the rules, afterward we can use these rules (often called model) to predict the output for not seen inputs. For example, it can categorize our images and predict if there is a cat or dog in the picture.
In our example, it's a supervised algorithm, because we know what the output should be and feed the target values into our algorithm.
In our very simplified example, we have a fixed set of inputs and only two outcomes (categories).
Now, let’s take a look at what does it mean when we say the algorithm learns something.
To learn something we need parameters that can be optimized. In the first picture above we have a thinking bubble in the brain of our robot, let’s take a closer look at what happens in this bubble.
Our little robot has a small neuronal network with only one neuron (blue circle) as a brain because his only responsibility at the moment is taking care of our plant.
First, our neuron gets the inputs, let's say [1, 1] enough water and sun and then the algorithm calculates first (1) the weighted sum, it sums up the multiplication of w (weight) and x (input) as many times as in n specified started at i = 0. You can see an example on the left.
And second (2) it checks with the output of the weighted sum= -1 (the output of the example) if the neuron should get activated in our example we use the simple Heaviside function it returns 1 if the output of weighted sum is above 0 and if bellow it returns 0. This means our little robot will take action if the weighted sum is above 0 and otherwise it waits, with -1 given our robot will wait.
Now let's code this in plain JavaScript.
In the weights and input array, you will not just see the two values for water and sun, you see the third parameter at the beginning. That's the bias.
A bias value allows you to shift the activation function to the left or right.
You can read more about the bias here.
First, we will add the Perceptron class with the empty functions to get a clear picture of the structure.
class Perceptron {
constructor() {
// (1) initialization
} _heaviside(weightedSum) {
// (2) activation
} _calculateWeightedSum(input) {
// (3) calculates the weighted sum
} predict(input) {
// (4) prediction
} fit(iterations, trainingData) {
// (5) trains our algorithm and updates our weigths
} _updateWeights(error, input, target) {
// (6) updates our class property weights
}
}
(1) We initialize our weights, you can choose any other numbers if you wish.
constructor() {
this._weights = [0, 0, 0];
}(2) We add the activation function. It simply returns 1 if the weightedSum is above 0 otherwise it returns 0.
_heaviside(weightedSum) {
return weightedSum >= 0 ? 1 : 0;
}(3) First, we check if our class property weights have the same length as the input and if so we iterate over each input and calculate the weighted sum. At the end of the function we return it, it will always be a number.
_calculateWeightedSum(input) {
let weightedSum = 0;
if (this._weights.length === input.length) {
input.forEach((x, i) => (weightedSum += x * this._weights[i]));
} else {
console.error("Length of weights and inputs doesnt match");
} return weightedSum;
}
(4) We add the prediction function that calculates the weighted sum and with the result of that checks if the neuron should get activated with the Heaviside function.
predict(input) {
const weightedSum = this._calculateWeightedSum(input);
return this._heaviside(weightedSum);
}(5) Within the fit function we train our algorithm. We have as many learning iterations as specified in the iterations parameter, as the second parameter we get the prepared training data.
We initialize errors with an empty array so we can, later on, check how many errors we got.
We want to get a random input set of our trainingData, because of this we set a random index into randomIndexOfTrainingData and afterward select that input. This is important if you have large data set.
The variable selectedTrainingData holds an object with the properties input for example [1, 1, 0] and target 1 we make a prediction on that value then calculate the error this simply means we calculate the difference between target and calculatedTarget and check if our prediction was correct.
Then we update the class property weights.
fit(iterations, trainingData) {
let errors = []; for (let i = 0; i < iterations; i++) {
const randomIndexOfTrainingData = Math.floor(
Math.random() * trainingData.length
);
const selectedTrainingData = trainingData[randomIndexOfTrainingData]; const input = selectedTrainingData.input;
const target = selectedTrainingData.target; const calculatedTarget = this.predict(input, target);
const error = target - calculatedTarget; this._updateWeights(error, input, target);
errors.push(error);
}
return {errors, weights: this._weights};
}
(6) Finally, we add the update weights function to our Perceptron class that iterates over the input for example [1, 1, 0], and updates the class property weights if there was an error.
_updateWeights(error, input, target) {
input.forEach((x, i) => {
this._weights[i] = this._weights[i] + error * x;
})
}Now let's see our algorithm in action.
We define our training data with the input values and the target values. The first value is the bias in the input property. The second and third values are from our sun and water sensors.
const trainingData = [
{
input: [1, 0, 0],
target: 1
},
{
input: [1, 0, 1],
target: 1
},
{
input: [1, 1, 0],
target: 1
},
{
input: [1, 1, 1],
target: 0
},
];We create an instance of our Perceptron class.
const perceptron = new Perceptron();And start our learning process, I chose 30 iterations and pass in the traningData we specified above.
const result = perceptron.fit(30, trainingData);Then we make some predictions and log the results to the console.
console.log('result', result);// parameters, string for identification, prediction, target value
console.log('prediction 1', perceptron.predict([1, 0, 0]), 1);
console.log('prediction 2', perceptron.predict([1, 0, 1]), 1);
console.log('prediction 3', perceptron.predict([1, 1, 0]), 1);
console.log('prediction 4', perceptron.predict([1, 1, 1]), 0);
You can see the final code here.
I hope you had as much fun reading it as I had preparing to write this post. If there are open questions or feedback you are very welcome to leave a comment.
More content at plainenglish.io
