Skip to content

Latest commit

 

History

History
67 lines (45 loc) · 2.68 KB

readme.md

File metadata and controls

67 lines (45 loc) · 2.68 KB
Raw

Readme

I am keeping upgrading this, so the code may change everyday.

Environment

  • windows 10 (We use win32 API to operate the little knight and get screenshots)
  • python 3.8.8
  • python liberary: find in requirments.txt
  • Hollow Knight
  • HP Bar mod for Hollow Knight (In order to get the boss hp to calculate the reward, please find the mod in ./hollow_knight_Data/, and then copy the mod file to the game folder)
  • CUDA and cudnn for tensorflow

Usage

  • Now I only write train.py but not test.py (the file is just test some base functions not for model), you can write it by yourself if you get a good model.
  • I upload a saving file, if you never played this game, please move /save_file/user3.dat into save folder (usually C:\user\_username_\AppData\LocalLow\Team Cherry\Hollow Knight)
  • Adjust the game resolution to 1920*1017
  • Run train.py
  • Keep the game window at the forefront (Since I cannot send keyboard event in the background, I tried PossMassage() in win32 API, but it did not work well. If you have any idea about sending keyboard event in the background, please let me know)
  • Let the little knight stand in front of the statue of the boss in the godhome
  • Press F1 to start trainning. (Also you can use F1 to stop trainning)

Code structure

  • Most training configuration is in train.py

  • Agent.py gets output actions from our model

  • DQN.py is the learning algorithm

  • Model.py defines the model we use

  • ReplayMemory.py defines the experience pool for learning

  • test.py is useless, I use it to test basic functions and fix bugs

  • Files in ./Tool are for other functions we may use

  • Actions defines actions for little knight and restart game script

  • GetHp help us get our hp, boss hp, soul and location(it may have some bugs, you can fix it by yourself)

  • SendKey is the API we use to send keyboard event to windows system.

  • UserInput is an useless file, which I used it to train my model manually.

  • WindowsAPI is used to get screenshot of the game, and key_check() is used to check which key is pressed.

  • Helper defines [Reward Jugment] fucntion, and other functions we may use

Changes

  • Add delay reward of an action

  • Make the mdoel output an action sequence

  • Use two models to output actions. One is for moving and the other is for attack/jump/skill

  • Apply RESNET

  • Add LSTM layers

  • Merge a part of action model and move model

  • Remvoe LSTM layers

  • No more Q value, reward is enough, Q value is too complex to learn.

  • Use kernel32 to read player Hp and hornet Hp.

  • Use different criteria to evaluate move and actions.

  • Do not use skill without souls

  • Use more precise scoring standards