Training an artificial neural network for the micro of marines

Anything related on how to build / improve your AI using BWAPI, BWTA, BWSAL, etc.
User avatar
Leckofunny
Posts: 27
Joined: Mon Apr 04, 2016 7:41 pm

Re: Training an artificial neural network for the micro of marines

Postby Leckofunny » Thu Jun 16, 2016 3:09 pm

I just fixed a tremendously bug. My InpuVector object was just existing once and I passed just its reference to the individual units. So the units would work with information of other units. This mixed up the training data a lot. So this issue is present since the very first training attempts.

Hopefully I get now proper training examples.
User avatar
Leckofunny
Posts: 27
Joined: Mon Apr 04, 2016 7:41 pm

Re: Training an artificial neural network for the micro of marines

Postby Leckofunny » Thu Jun 16, 2016 8:22 pm

It looks like that I can completely focus on the fitness function. After gathering data for 100 matches one of the two trained neural nets turned out with this behavior:

https://www.youtube.com/watch?v=-2Mj-W8 ... e=youtu.be

The other one would be completely stuck on moving back.
User avatar
Leckofunny
Posts: 27
Joined: Mon Apr 04, 2016 7:41 pm

Re: Training an artificial neural network for the micro of marines

Postby Leckofunny » Thu Jul 07, 2016 8:43 am

Here are some news.

During the last weeks I fixed a bug in my fitness function which prevented punishing moving back accordingly. Now it is easy to reproduce trained behaviors. It usually takes 35 matches (15 minutes total) to train a neural net given the underlying training scenario.

The neural net structure, which I used originally, was oversized in comparison to the generated training examples. Shrinking down the hidden node count from 40 to 28 helped the neural net to solve unknown situations. An untrained situation are pushing in enemies, who just run into the squad. Due to overfitting the neural net would decide to run away. With less hidden nodes the neural net attacks the enemies who try to cut the lines. Basically if a neural net is oversized, it behaves like a big look up table, which only allows to solve the trained examples.

Concerning my thesis, I'm pretty close to submission. All what's left is to polish some things and to print the thesis. This is a playlist of some of the results:
https://www.youtube.com/watch?v=gKgcAAF ... Qa3-K2nRQv
krasi0
Posts: 254
Joined: Thu Dec 18, 2008 12:42 pm

Re: Training an artificial neural network for the micro of marines

Postby krasi0 » Thu Jul 07, 2016 8:59 am

Congratulations on the registered progress and on finding that nasty bug!
Do you think the approach would scale well to mixed unit type compositions, e.g. Hydralisks + Lurkers + Queens against Marines + Medics + Siege Tanks?
User avatar
Leckofunny
Posts: 27
Joined: Mon Apr 04, 2016 7:41 pm

Re: Training an artificial neural network for the micro of marines

Postby Leckofunny » Thu Jul 07, 2016 9:15 am

My approach considers the immediate actions and their immediate outcomes. The BWAPI has some ugly constraints concerning attacks. It is not known when a projectile or strike is launched, neither is it possible to get feedback about which unit caused the damage. I rely on the assumption that every attack is successful, which is easy to implement for a scenario with one unit type only. With more varying units you'd have to predict the damage output. At least the kill count of an unit helps to get feedback if the unit managed to kill a foe. With these constraints, the implementation gets quite tedious. It is also very difficult to build a frame-perfect behavior, because the point of time for the strike or projectile to be launched is inconsistent and is not provided by the BWAPI.

It is not impossible and the approach itself is great for StarCraft, it's just that the implementation is a pain in the ass due to the constraints of the BWAPI. For the future, it would be awesome to have a tool, which allows the developer to setup training or testing sequences. Adding varying maps and AIs to such a training queue could test the quality of the neural net and to expose more varying situations to the units. Right now, messing with the Chaoslauncher, the config files and the dlls is painful as well.

It is possible, but it's lots of work. If the BWAPI gets more powerful concerning the issues I had, then I'd recommend to implement neural net behavior for StarCraft.


I'll send the thesis to you as soon as it gets done. My written conclusion pretty much covers the weaknesses and future perspective of my approach in more detail.
krasi0
Posts: 254
Joined: Thu Dec 18, 2008 12:42 pm

Re: Training an artificial neural network for the micro of marines

Postby krasi0 » Fri Oct 21, 2016 8:51 am

Hey! Has there been any recent developments with your ANNs work?
User avatar
Leckofunny
Posts: 27
Joined: Mon Apr 04, 2016 7:41 pm

Re: Training an artificial neural network for the micro of marines

Postby Leckofunny » Fri Oct 21, 2016 8:58 am

I stopped working on StarCraft since the BWAPI lacks in information concerning the knowledge of who attacked and when actually an attack action is done.

Now I'm working on my own experimental subject for conducting learning AI research. I wrote an Exposé for this concern https://drive.google.com/open?id=0BzzjW ... VVhenRONmM
Basically I want to have full control of the source code of the game while implementing a process for easily developing AI. I'm also developing tools for this game to add more comfort to automate matches and especially create training and testing match sequences,. It's pretty important to establish tests to measure the quality of the trained AI.


An other game which will use learning AI in the close future is Tuebor. The Tuebor develoeprs (Strength in Numbers) told me that they are using Markov Network Brains http://adamilab.msu.edu/markov-network-brains/
This is quite interesting, that's why I shared this link.
krasi0
Posts: 254
Joined: Thu Dec 18, 2008 12:42 pm

Re: Training an artificial neural network for the micro of marines

Postby krasi0 » Thu Oct 27, 2016 11:01 am

If something is lacking from BWAPI, you could either implement it on top of BWAPI or open a github issue. :)
User avatar
Leckofunny
Posts: 27
Joined: Mon Apr 04, 2016 7:41 pm

Re: Training an artificial neural network for the micro of marines

Postby Leckofunny » Thu Oct 27, 2016 2:12 pm

The attack issue is very well known already.
I've got other goals to achieve, so I'm done with StarCraft for now.
krasi0
Posts: 254
Joined: Thu Dec 18, 2008 12:42 pm

Re: Training an artificial neural network for the micro of marines

Postby krasi0 » Fri Feb 10, 2017 8:27 am

That's too bad. :(
What you were working on looked really promising!

Return to “AI development (BWAPI Bots)”

Who is online

Users browsing this forum: No registered users and 0 guests