Training an artificial neural network for the micro of marines

[Leckofunny]

I just fixed a tremendously bug. My InpuVector object was just existing once and I passed just its reference to the individual units. So the units would work with information of other units. This mixed up the training data a lot. So this issue is present since the very first training attempts.

Hopefully I get now proper training examples.

[Leckofunny]

It looks like that I can completely focus on the fitness function. After gathering data for 100 matches one of the two trained neural nets turned out with this behavior:

https://www.youtube.com/watch?v=-2Mj-W8 ... e=youtu.be

The other one would be completely stuck on moving back.

[Leckofunny]

Here are some news.

During the last weeks I fixed a bug in my fitness function which prevented punishing moving back accordingly. Now it is easy to reproduce trained behaviors. It usually takes 35 matches (15 minutes total) to train a neural net given the underlying training scenario.

The neural net structure, which I used originally, was oversized in comparison to the generated training examples. Shrinking down the hidden node count from 40 to 28 helped the neural net to solve unknown situations. An untrained situation are pushing in enemies, who just run into the squad. Due to overfitting the neural net would decide to run away. With less hidden nodes the neural net attacks the enemies who try to cut the lines. Basically if a neural net is oversized, it behaves like a big look up table, which only allows to solve the trained examples.

Concerning my thesis, I'm pretty close to submission. All what's left is to polish some things and to print the thesis. This is a playlist of some of the results:
https://www.youtube.com/watch?v=gKgcAAF ... Qa3-K2nRQv

[krasi0]

Congratulations on the registered progress and on finding that nasty bug!
Do you think the approach would scale well to mixed unit type compositions, e.g. Hydralisks + Lurkers + Queens against Marines + Medics + Siege Tanks?

[Leckofunny]

My approach considers the immediate actions and their immediate outcomes. The BWAPI has some ugly constraints concerning attacks. It is not known when a projectile or strike is launched, neither is it possible to get feedback about which unit caused the damage. I rely on the assumption that every attack is successful, which is easy to implement for a scenario with one unit type only. With more varying units you'd have to predict the damage output. At least the kill count of an unit helps to get feedback if the unit managed to kill a foe. With these constraints, the implementation gets quite tedious. It is also very difficult to build a frame-perfect behavior, because the point of time for the strike or projectile to be launched is inconsistent and is not provided by the BWAPI.

It is not impossible and the approach itself is great for StarCraft, it's just that the implementation is a pain in the ass due to the constraints of the BWAPI. For the future, it would be awesome to have a tool, which allows the developer to setup training or testing sequences. Adding varying maps and AIs to such a training queue could test the quality of the neural net and to expose more varying situations to the units. Right now, messing with the Chaoslauncher, the config files and the dlls is painful as well.

It is possible, but it's lots of work. If the BWAPI gets more powerful concerning the issues I had, then I'd recommend to implement neural net behavior for StarCraft.


I'll send the thesis to you as soon as it gets done. My written conclusion pretty much covers the weaknesses and future perspective of my approach in more detail.

[krasi0]

Hey! Has there been any recent developments with your ANNs work?

[Leckofunny]

I stopped working on StarCraft since the BWAPI lacks in information concerning the knowledge of who attacked and when actually an attack action is done.

Now I'm working on my own experimental subject for conducting learning AI research. I wrote an Exposé for this concern https://drive.google.com/open?id=0BzzjW ... VVhenRONmM
Basically I want to have full control of the source code of the game while implementing a process for easily developing AI. I'm also developing tools for this game to add more comfort to automate matches and especially create training and testing match sequences,. It's pretty important to establish tests to measure the quality of the trained AI.


An other game which will use learning AI in the close future is Tuebor. The Tuebor develoeprs (Strength in Numbers) told me that they are using Markov Network Brains http://adamilab.msu.edu/markov-network-brains/
This is quite interesting, that's why I shared this link.

[krasi0]

If something is lacking from BWAPI, you could either implement it on top of BWAPI or open a github issue.

[Leckofunny]

The attack issue is very well known already.
I've got other goals to achieve, so I'm done with StarCraft for now.

[krasi0]

That's too bad.
What you were working on looked really promising!