Game Development

Algorithm of machine learning Flappy Bird

I will introduce you to the full tutorial on HTML5 with a demo on the algorithm of machine learning the video game Flappy Bird. The purpose of this experiment is to write an artificial intelligence controller based on neural networks and a genetic algorithm.

That is, we want to create an AI robot that can learn the optimal game in Flappy Bird. As a result, our little bird can safely fly through obstacles. In the best scenario, she will never die.

After reading the theory behind this project, you can download the source code at the end of this tutorial. All code is written in HTML5 using the <a title="Phaser Framework" href=""> phaser </a> framework. In addition, we used the Synaptic Neural Network library to implement a neural network so that it would not be created from scratch.
First, look at the demo to evaluate the algorithm in action:


Run in full-screen mode
Video presentation
In addition to the demo, you can watch a short video with a simple explanation of the algorithm. Like those who like to receive information quickly!
What is the "machine learning algorithm"
According to Arthur Samuel's formulation of 1959, machine learning is a way to make computers work without programming in explicit form. In general, this is a process of fine tuning of learning, gradually improving the original random system.

That is, the goal here is to create an artificial intelligence that can find the right solution from a bad system by fine tuning the model parameters. For this, the algorithm of machine learning uses a variety of different approaches.

Specifically in this project, the basic approach to machine learning (ML) is based on neuroevolution. In this form of machine learning, evolutionary algorithms, such as the genetic algorithm (GA), are used to teach artificial neural networks (ANN).

That is, in our case we can say that ML = GA + ANN.
Artificial neural network
An artificial neural network is a subset of the machine learning algorithm. As a basis, the structure and functions of biological neural networks are taken. These networks are created from a set of neurons that transmit signals to each other.

That is, to create an artificial brain, we need to simulate neurons and connect them, so that they form a neural network.

A standard artificial neural network consists of a layer of input data, one or more hidden layers and a layer of output data. In each layer there are several neurons. Neurons of input and output data are attached directly to the external environment. Hidden neurons are connected between them.

In this project, each object (bird) has its own neural network, used as an AI brain for the passage of the game. It consists of the following three layers:

a layer of input data with two neurons represents what the bird sees:

horizontal distance to the nearest gap
height difference with the nearest interval

hidden layer with six neurons
A layer of output data with one neuron, creating an action:
if the output is> 0.5, then make a jerk, otherwise do not do anything

The figure below shows the neural network architecture for this example:

The genetic algorithm
When we discussed the algorithm of machine learning, it was said that the genetic algorithm is used to train and improve neural networks.

The genetic algorithm is a search-based optimization technique that replicates natural selection and genetics. It uses the same combination of selection, crossing-over and mutations to change the initial random population.

Here are the main stages of the implementation of our genetic algorithm:

create an initial population of 10 objects (birds) with random neural networks
We let all objects play simultaneously using their own neural networks
for each object we calculate its fitness function to evaluate its quality (for more details, see the fitness function section)
after the death of all objects, we evaluate the current generation to create a new one with the help of genetic operators (for more details, see the Replacement Strategy section)
back to step 2
Fitness function
In addition to the genetic algorithm (step 3), we will consider here in more detail the fitness function - what it is and how to define it.

Since we want the population to evolve from the best objects, we need to define the fitness function.

In general, the fitness function is a metric that measures the quality of an object. If we have the quality metric of each bird, we can select the most adapted objects and use them to recreate the next generation.

In this project, we reward the bird in direct proportion to the distance traveled. In addition, we punish it by the current distance to the nearest interval. In this way, we can distinguish between birds flying the same distance.

Let's sum up: our fitness function is the difference between the total distance done by the bird and the current distance to the nearest interval.

Replacement strategy
In addition to the genetic algorithm (step 4), here are the stages of applying natural evolution to a dying generation. In fact, the best objects survive, and their descendants replace the worst objects as follows:

sort the objects of the current generation according to their level of fitness
choose the four best objects (winners) and transfer them directly to the next generation
We create one descendant as the result of crossing-over between the two best winners
We create three descendants as the results of crossing-over of two random winners
create two descendants as direct copies of two random winners
we apply random mutations to each descendant to add variability
Source code
And finally, the link for downloading the source code:
Algorithm of machine learning for Flappy Bird - conclusion
In this tutorial, we successfully implemented the AI ​​robot to teach the game in Flappy Bird. As a result of several iterations, we can get an almost invulnerable player. To achieve this goal, we used two approaches to machine learning algorithms: artificial neural networks and a genetic algorithm.

For the experiment, you can try to change some of the parameters in the code and see what happens. For example, you can change the number of neurons in a hidden layer or the number of objects in a population. In addition, you can try to change fitness functions. Moreover, try to change some physical parameters - distances between obstacles, gravity and so on.

Try also to apply the same approach to evolution in other games.
MeLavi 10 october 2017, 9:05
Vote for this post
Bring it to the Main Page


Leave a Reply

Avaible tags
  • <b>...</b>highlighting important text on the page in bold
  • <i>..</i>highlighting important text on the page in italic
  • <u>...</u>allocated with tag <u> text shownas underlined
  • <s>...</s>allocated with tag <s> text shown as strikethrough
  • <sup>...</sup>, <sub>...</sub>text in the tag <sup> appears as a superscript, <sub> - subscript
  • <blockquote>...</blockquote>For  highlight citation, use the tag <blockquote>
  • <code lang="lang">...</code>highlighting the program code (supported by bash, cpp, cs, css, xml, html, java, javascript, lisp, lua, php, perl, python, ruby, sql, scala, text)
  • <a href="http://...">...</a>link, specify the desired Internet address in the href attribute
  • <img src="http://..." alt="text" />specify the full path of image in the src attribute