GTA AI

School project: Making an AI that drives a car in GTA San Andreas (Possible to apply to other similar games)

Using python with OpenCV and tenserflow as our machine learning backbone.

Installation

Prerequisites

DxWind

Used to run the game in windowed mode, as the game doesnt support it naturally

When you set up DxWind it should look something like this

GTA:SA

You also need a working copy of Grand Theft Auto: San Andreas

Image recognition and machine learning

The code was compiled using python 3.6.

Tenserflow and tflearn are also required as our machine learning backbone.

Opencv is a must as almost 70% of the files use it in some way.

Unfortinatly as of now opencv doesnt support python 3 officaly, so here is a temporary workaround.

• Download the correct version from this site

  

• Or use the file provided in the repo

• Run this command

  pip install opencv_python-3.4.1+contrib-cp35-cp35m-win_amd64.whl
  

• Now you should be able to import opencv without any errors

Usage

First step is to collect data, which will be used for training our model. You can do so by running data_creation.py

python data_creation.py -h #To list out all the possible arguments

This will monitor your keypresses and capture your game screen and write them down in a file.

The format for the keypresses is as follows.

  type 1
  #a w s d
  [0,0,0,0] #one for pressed otherwise zero
  type 2
  #a w d
  [0,0,0] #one for pressed otherwise zero

You can also press p to pause recording of data

After that we need to balance the data for our CNN

You do so by running balancing_data.py

python balancing_data.py --save somepath --path otherpath 

Where

--save Specify the path where your file will be saved after balancing*
--path Path of the file to be balanced

You should get an output similar to this (Where the 3/4 numbers are indentical to each other)

For training the model use train_model.py Which has the following arguments

-w,    --width      # Specifies the width and the height
--ht,  --heigth     # of the image that will be fed to the CNN
--lr                # Learning rate
-e,    --epochs     # Number of learning iterations
--save --save_path  # The path where the model will be saved

For testing the model launch GTA:SA with DxWind And run test_model.py with the following arguments

-w,    --width      # Specifies the width and the height
--ht,  --heigth     # of the image that will be fed to the CNN
--lr                # Learning rate
-e,    --epochs     # Number of learning iterations
--fw,  --fw-th      # Forward threshold, should be between 1 and 0
'-s', '--s-th'      # Sides threshold, should be between 1 and 0

The model.predict method inside the code returns probabilites of each of the keys being pressed

So we need the threshold to determine when we are sure we need to simulate a keypress

Misc

Useful cheat codes

Cheat codes(ingame) for ease of debugging

aezakmi         -> No wanted level
hesoyam         -> Health, money and repair car
pleasantlywarm  -> sunny waether (bugs on foggy, rainy)
nightprowler    -> Night time (might work better this way)
slowitdown      -> Slower game (Debugging/Early development purposes). This one stacks!! Use speeditup to escape
speeditup       -> Faster gameplay

Models and data

Here are some pretrained models and prerecorded data

https://drive.google.com/open?id=1zgaTgFHkm-igtfkfF7bzOvt-9JKi8e4A