[Question] ML algorithms to design better AI

[Killbrum]

Good time of the day, folks!
I think that nowadays it might be really interesting to design "another wheel" using Machine Learning. Of course, we will need to develop a specific algorithm for exploring, battle, magic casts, castle improvement and etc. But mainly it might be a very good experience since this game is not so big. What do you think?

[ihhub]

Hi @Killbrum ,

If you move apart all hype of these fancy words about machine learning we should answer these questions:

• what are benefits of using probability theory algorithms in comparison with current logic?
• what modules of logic should be used for machine learning?
• how many resources does the game will draw for the algorithms? How well will it work on low profile devices?
• which framework is going to be used for this? How heavy is it? How does it impact development and compilation?
• what's the measure of efficiency of algorithms? Do we need to use some test maps or cases?

Of course we could implement it but a proper measure of resources (in our case hours of developers and QA teams) and the output of this solution is very essential.

[ihhub]

However, our code design allows us to add a new AI without impacting existing one so if someone wants to try their skills we don't see an issue.

The proper solution with AI what we see right now is an implementation of prediction algorithms where we could simulate the outcome of moves on a map by a hero to find the most optimal path or calculate all possible outcomes for a few turns in battles. It's a brute force algorithm but it's totally valid in such small scale game.

[idshibanov]

We already entertained the idea of using ML, but we have to define better how we're going to use it. Current implementation (far from being done) relies heavily on calculating heuristics. Simplest approach would be to record actions and outcomes to build a learning set and eventually adjust heuristics based on values you've got.

Or as @ihhub said, you can create a second version of AI code and experiment there.

[ihhub]

As a conclusion: yes, we're totally open for it if the project will benefit but we need to be rational in implementation and resource management :)

[Killbrum]

Yes, I'm really glad that I heard the feedback, and we at least started this topic :)

For sure, there are plenty of questions that should be answered. Moreover, this "feature" isn't for 1.0 or even 1.2. I would say that it is much closer to 2.0 release, like long term plans, nothing more.

Let me try to explain my vision and problems that I would like to solve.

1. Game perspective. People usually struggle with stupid AI. It's not hard to beat impossible AI (200%). Actually, if I recall correctly, AI doesn't behave differently starting from hard (130%). It just receives more resources. So the main idea is to make way more interesting games with a computer. For sure, we can try to improve our current AI, and we need to! But, at the same time, it might be incredibly hard to make really interesting AI. I just want to say that there are plenty of cases, tricks, and "hacks" that might be useful in the game. For example, AI needs to understand the difference between Lightning and Cold Ray and use them correctly with different artifacts and different conditions. There might be different tactics for playing on different map sizes or what you need to build based on explored resources, near situation, discovered enemies, etc. Mainly it's not just hard. It might be a development for years and require really excellent in-game expertise. With Machine Learning, we might be able to set rules for an algorithm and give him a chance to explore the world by himself.

2. Development perspective. I'm pretty sure that we will see new interesting solutions in the near future. It's just a matter of time. Since this game is pretty small and already has working AI and a lot of well-known tips and tricks, it might be relatively simple to develop new AI

[ihhub]

Hi @Killbrum , our AI is developed from scratch, mostly by @idshibanov so it's not the same as in the original game. We have many issues with AI logic and we know them but we're very limited with resources, especially with developers.

To develop AI with machine learning we need to prepare the current AI to support it. In my view we have to fix and polish existing AI with main focus on battles. Next step is to add code to simulate multiple scenarios like moving hero on map or possible movements in battle per turn. We have some command based prototyped code for this. After that having such code allows us to train a model based of some coefficients which we still need to define. Also we will have minimum 2 models: one for battles and one for map exploration and kingdom development.

That's why I asked so many questions not to cancel the idea but to have a clear vision of development.

[Killbrum]

Yep, you're right. I was poking around last year. I saw this project too many times, and actually, HoMM 2 is my favorite. But I used to have my own project (MMORPG emulator of one Korean game). Due to unpredictable blockers, I'm not able to continue my own stuff anymore, so I would like to use my free pair of hands right here :)

Okay, it's really good that you're open, and I will try to dig deeper, collect information, find possible ways to resolve this and get back to you. Of course, it will take a while since milestones 0.9 and 1.0 are much more important.

[ihhub]

Converting this issue into a discussion as it goes beyond the scope of the project.