The sketch.js file precisely mirrors the steps of the GA. However, most of the functionality called upon is encapsulated in the DNA class.
class DNA {
+ //{!7} Constructor (makes a random DNA)
constructor(length) {
this.genes = [];
for (let i = 0; i < length; i++) {
@@ -705,6 +705,7 @@ Example 9.1: Gene
let c = floor(random(32, 127));
return String.fromCharCode(c);
}
+In Example 9.1, you might notice that new child elements are directly added to the population array. This approach is possible because I have a separate mating pool array that contains references to the original parent elements. However, if I were to instead use the relay-race weightedSelection() function, I’d need to create a temporary array for the new population. This temporary array would hold the child elements and replace the original population array only after the reproduction step is completed. You’ll see this implemented in Example 9.2.
Exercise 9.6
diff --git a/content/10_nn.html b/content/10_nn.html index c9b66095..87e18eaa 100644 --- a/content/10_nn.html +++ b/content/10_nn.html @@ -331,10 +331,8 @@The Perceptron Code
train(inputs, desired) { // Step 2: Guess according to those inputs. let guess = this.feedforward(inputs); - // Step 3: Compute the error (the difference betweendesired and guess).
let error = desired - guess;
-
//{!3} Step 4: Adjust all the weights according to the error and learning constant.
for (let i = 0; i < this.weights.length; i++) {
this.weights[i] = this.weights[i] + error * inputs[i] * this.learningConstant;
@@ -805,7 +803,7 @@ Training the Model
console.log("Training complete!"); }Yes, that’s it! After all, the hard work has already been completed. The data was collected, prepared, and fed into the model. All that remains is to call the train() method, sit back, and let ml5.js do its thing.
In truth, it isn’t quite that simple. If I were to run the code as written and then test the model, the results would probably be inadequate. Here’s where another key term in machine learning comes into play: epochs. The train() method tells the neural network to start the learning process. But how long should it train for? You can think of an epoch as one round of practice, one cycle of using the entire training dataset to update the weights of the neural network. Generally speaking, the more epochs you go through, the better the network will perform, but at a certain point you’ll have diminishing returns. The number of epochs can be set by passing in an options object into train():
In truth, it isn’t quite that simple. If I were to run the code as written and then test the model, the results would probably be inadequate. Here’s where another key term in machine learning comes into play: epochs. The train() method tells the neural network to start the learning process. But how long should it train for? You can think of an epoch as one round of practice, one cycle of using the entire training dataset to update the weights of the neural network. Generally speaking, the more epochs you go through, the better the network will perform, but at a certain point you’ll have diminishing returns. The number of epochs can be set by passing in an options object into train().
//{!1} Set the number of epochs for training.
let options = { epochs: 25 };
classifier.train(options, finishedTraining);
diff --git a/content/11_nn_ga.html b/content/11_nn_ga.html
index 7c124568..7f5d320a 100644
--- a/content/11_nn_ga.html
+++ b/content/11_nn_ga.html
@@ -46,9 +46,6 @@ Reinforcement Learning
- The x-distance to the next pipe
These features are illustrated in Figure 11.2.
@@ -72,7 +69,7 @@Reinforcement Learning
inputs: 5, outputs: ["flap", "no flap"], task: "classification" -} +}; let birdBrain = ml5.neuralNetwork(options);What next? If I were following the steps I laid out in Chapter 10, I’d have to go back to steps 1 and 2 of the machine learning process: data collection and preparation. How exactly would that work here? One idea could be to scour the earth for the greatest Flappy Bird player of all time and record them playing for hours. I could log the input features for every moment of gameplay along with whether the player flapped or not. Feed all that data into the model, train it, and I can see the headlines already: “Artificial Intelligence Bot Defeats Flappy Bird.”
But wait a second; has a computerized agent really learned to play Flappy Bird on its own, or has it simply learned to mirror the gameplay of a human? What if that human missed a key aspect of Flappy Bird strategy? The automated player would never discover it. Not to mention that collecting all that data would be incredibly tedious.
@@ -94,7 +91,6 @@Coding Flappy Bird
// The bird’s position (x will be constant)
this.x = 50
this.y = 120;
-
// Velocity and forces are scalar since the bird moves only along the y-axis.
this.velocity = 0;
this.gravity = 0.5;
@@ -112,7 +108,6 @@ Coding Flappy Bird
this.y += this.velocity; // Dampen velocity. this.velocity *= 0.95; - // Handle the floor. if (this.y > height) { this.y = height; @@ -278,14 +273,14 @@The Bird Brain
}The neural network’s prediction is in the same format as the gesture classifier from Chapter 10, and the decision can be made by checking the first element of the results array. If the output label is "flap", then call flap().
- Now that I’ve finished the think() method, the real challenge can begin: teaching the bird to win the game by consistently flapping its wings at the right moment. This is where the GA comes back into the picture. Recalling the discussion from Chapter 9, three key principles underpin Darwinian evolution: variation, selection, and heredity. I’ll revisit each of these principles in turn as I implement the steps of the GA in this new context of neural networks.
Variation: A Flock of Flappy Birds
A single bird with a randomly initialized neural network isn’t likely to have any success at all. That lone bird will most likely jump incessantly and fly way off-screen, or sit perched at the bottom of the canvas awaiting collision after collision with the pipes. This erratic and nonsensical behavior is a reminder: a randomly initialized neural network lacks any knowledge or experience. The bird is essentially making wild guesses for its actions, so success is going to be rare.
This is where the first key principle of GAs comes in: variation. The hope is that by introducing as many different neural network configurations as possible, a few might perform slightly better than the rest. The first step toward variation is to add an array of many birds (Figure 11.4).
+
+ // Population size let populationSize = 200; // Array of birds @@ -343,7 +338,6 @@Selection: Flappy Bird Fitness
// Update and show the bird. bird.update(); bird.show(); - //{!4} Has the bird hit a pipe? If so, it’s no longer alive. for (let pipe of pipes) { if (pipe.collides(bird)) {