School project: Making an AI that drives a car in GTA San Andreas (Possible to apply to other similar games)
Using python with OpenCV and tenserflow as our machine learning backbone.
Used to run the game in windowed mode, as the game doesnt support it naturally
When you set up DxWind it should look something like this
GTA:SA
You also need a working copy of Grand Theft Auto: San Andreas
Image recognition and machine learning
The code was compiled using python 3.6.
Tenserflow and tflearn are also required as our machine learning backbone.
Opencv is a must as almost 70% of the files use it in some way.
Unfortinatly as of now opencv doesnt support python 3 officaly, so here is a temporary workaround.
-
Download the correct version from this site
-
Or use the file provided in the repo
-
Run this command
pip install opencv_python-3.4.1+contrib-cp35-cp35m-win_amd64.whl -
Now you should be able to import opencv without any errors
First step is to collect data, which will be used for training our model. You can do so by running data_creation.py
python data_creation.py -h #To list out all the possible arguments
This will monitor your keypresses and capture your game screen and write them down in a file.
The format for the keypresses is as follows.
type 1
#a w s d
[0,0,0,0] #one for pressed otherwise zero
type 2
#a w d
[0,0,0] #one for pressed otherwise zeroYou can also press p to pause recording of data
After that we need to balance the data for our CNN
You do so by running balancing_data.py
python balancing_data.py --save somepath --path otherpath
Where
--save Specify the path where your file will be saved after balancing*
--path Path of the file to be balanced
You should get an output similar to this (Where the 3/4 numbers are indentical to each other)
For training the model use train_model.py Which has the following arguments
-w, --width # Specifies the width and the height
--ht, --heigth # of the image that will be fed to the CNN
--lr # Learning rate
-e, --epochs # Number of learning iterations
--save --save_path # The path where the model will be saved
For testing the model launch GTA:SA with DxWind And run test_model.py with the following arguments
-w, --width # Specifies the width and the height
--ht, --heigth # of the image that will be fed to the CNN
--lr # Learning rate
-e, --epochs # Number of learning iterations
--fw, --fw-th # Forward threshold, should be between 1 and 0
'-s', '--s-th' # Sides threshold, should be between 1 and 0
The model.predict method inside the code returns probabilites of each of the keys being pressed
So we need the threshold to determine when we are sure we need to simulate a keypress
Cheat codes(ingame) for ease of debugging
aezakmi -> No wanted level
hesoyam -> Health, money and repair car
pleasantlywarm -> sunny waether (bugs on foggy, rainy)
nightprowler -> Night time (might work better this way)
slowitdown -> Slower game (Debugging/Early development purposes). This one stacks!! Use speeditup to escape
speeditup -> Faster gameplay
Here are some pretrained models and prerecorded data
https://drive.google.com/open?id=1zgaTgFHkm-igtfkfF7bzOvt-9JKi8e4A