I just fixed a tremendously bug. My InpuVector object was just existing once and I passed just its reference to the individual units. So the units would work with information of other units. This mixed up the training data a lot. So this issue is present since the very first training attempts.
Hopefully I get now proper training examples.
Training an artificial neural network for the micro of marines
- Leckofunny
- Posts: 27
- Joined: Mon Apr 04, 2016 7:41 pm [phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable
- Leckofunny
- Posts: 27
- Joined: Mon Apr 04, 2016 7:41 pm [phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable
Re: Training an artificial neural network for the micro of marines
It looks like that I can completely focus on the fitness function. After gathering data for 100 matches one of the two trained neural nets turned out with this behavior:
https://www.youtube.com/watch?v=-2Mj-W8 ... e=youtu.be
The other one would be completely stuck on moving back.
https://www.youtube.com/watch?v=-2Mj-W8 ... e=youtu.be
The other one would be completely stuck on moving back.
- Leckofunny
- Posts: 27
- Joined: Mon Apr 04, 2016 7:41 pm [phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable
Re: Training an artificial neural network for the micro of marines
Here are some news.
During the last weeks I fixed a bug in my fitness function which prevented punishing moving back accordingly. Now it is easy to reproduce trained behaviors. It usually takes 35 matches (15 minutes total) to train a neural net given the underlying training scenario.
The neural net structure, which I used originally, was oversized in comparison to the generated training examples. Shrinking down the hidden node count from 40 to 28 helped the neural net to solve unknown situations. An untrained situation are pushing in enemies, who just run into the squad. Due to overfitting the neural net would decide to run away. With less hidden nodes the neural net attacks the enemies who try to cut the lines. Basically if a neural net is oversized, it behaves like a big look up table, which only allows to solve the trained examples.
Concerning my thesis, I'm pretty close to submission. All what's left is to polish some things and to print the thesis. This is a playlist of some of the results:
https://www.youtube.com/watch?v=gKgcAAF ... Qa3-K2nRQv
During the last weeks I fixed a bug in my fitness function which prevented punishing moving back accordingly. Now it is easy to reproduce trained behaviors. It usually takes 35 matches (15 minutes total) to train a neural net given the underlying training scenario.
The neural net structure, which I used originally, was oversized in comparison to the generated training examples. Shrinking down the hidden node count from 40 to 28 helped the neural net to solve unknown situations. An untrained situation are pushing in enemies, who just run into the squad. Due to overfitting the neural net would decide to run away. With less hidden nodes the neural net attacks the enemies who try to cut the lines. Basically if a neural net is oversized, it behaves like a big look up table, which only allows to solve the trained examples.
Concerning my thesis, I'm pretty close to submission. All what's left is to polish some things and to print the thesis. This is a playlist of some of the results:
https://www.youtube.com/watch?v=gKgcAAF ... Qa3-K2nRQv
-
- Posts: 254
- Joined: Thu Dec 18, 2008 12:42 pm [phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable
Re: Training an artificial neural network for the micro of marines
Congratulations on the registered progress and on finding that nasty bug!
Do you think the approach would scale well to mixed unit type compositions, e.g. Hydralisks + Lurkers + Queens against Marines + Medics + Siege Tanks?
Do you think the approach would scale well to mixed unit type compositions, e.g. Hydralisks + Lurkers + Queens against Marines + Medics + Siege Tanks?
- Leckofunny
- Posts: 27
- Joined: Mon Apr 04, 2016 7:41 pm [phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable
Re: Training an artificial neural network for the micro of marines
My approach considers the immediate actions and their immediate outcomes. The BWAPI has some ugly constraints concerning attacks. It is not known when a projectile or strike is launched, neither is it possible to get feedback about which unit caused the damage. I rely on the assumption that every attack is successful, which is easy to implement for a scenario with one unit type only. With more varying units you'd have to predict the damage output. At least the kill count of an unit helps to get feedback if the unit managed to kill a foe. With these constraints, the implementation gets quite tedious. It is also very difficult to build a frame-perfect behavior, because the point of time for the strike or projectile to be launched is inconsistent and is not provided by the BWAPI.
It is not impossible and the approach itself is great for StarCraft, it's just that the implementation is a pain in the ass due to the constraints of the BWAPI. For the future, it would be awesome to have a tool, which allows the developer to setup training or testing sequences. Adding varying maps and AIs to such a training queue could test the quality of the neural net and to expose more varying situations to the units. Right now, messing with the Chaoslauncher, the config files and the dlls is painful as well.
It is possible, but it's lots of work. If the BWAPI gets more powerful concerning the issues I had, then I'd recommend to implement neural net behavior for StarCraft.
I'll send the thesis to you as soon as it gets done. My written conclusion pretty much covers the weaknesses and future perspective of my approach in more detail.
It is not impossible and the approach itself is great for StarCraft, it's just that the implementation is a pain in the ass due to the constraints of the BWAPI. For the future, it would be awesome to have a tool, which allows the developer to setup training or testing sequences. Adding varying maps and AIs to such a training queue could test the quality of the neural net and to expose more varying situations to the units. Right now, messing with the Chaoslauncher, the config files and the dlls is painful as well.
It is possible, but it's lots of work. If the BWAPI gets more powerful concerning the issues I had, then I'd recommend to implement neural net behavior for StarCraft.
I'll send the thesis to you as soon as it gets done. My written conclusion pretty much covers the weaknesses and future perspective of my approach in more detail.
-
- Posts: 254
- Joined: Thu Dec 18, 2008 12:42 pm [phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable
Re: Training an artificial neural network for the micro of marines
Hey! Has there been any recent developments with your ANNs work?
- Leckofunny
- Posts: 27
- Joined: Mon Apr 04, 2016 7:41 pm [phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable
Re: Training an artificial neural network for the micro of marines
I stopped working on StarCraft since the BWAPI lacks in information concerning the knowledge of who attacked and when actually an attack action is done.
Now I'm working on my own experimental subject for conducting learning AI research. I wrote an Exposé for this concern https://drive.google.com/open?id=0BzzjW ... VVhenRONmM
Basically I want to have full control of the source code of the game while implementing a process for easily developing AI. I'm also developing tools for this game to add more comfort to automate matches and especially create training and testing match sequences,. It's pretty important to establish tests to measure the quality of the trained AI.
An other game which will use learning AI in the close future is Tuebor. The Tuebor develoeprs (Strength in Numbers) told me that they are using Markov Network Brains http://adamilab.msu.edu/markov-network-brains/
This is quite interesting, that's why I shared this link.
Now I'm working on my own experimental subject for conducting learning AI research. I wrote an Exposé for this concern https://drive.google.com/open?id=0BzzjW ... VVhenRONmM
Basically I want to have full control of the source code of the game while implementing a process for easily developing AI. I'm also developing tools for this game to add more comfort to automate matches and especially create training and testing match sequences,. It's pretty important to establish tests to measure the quality of the trained AI.
An other game which will use learning AI in the close future is Tuebor. The Tuebor develoeprs (Strength in Numbers) told me that they are using Markov Network Brains http://adamilab.msu.edu/markov-network-brains/
This is quite interesting, that's why I shared this link.
-
- Posts: 254
- Joined: Thu Dec 18, 2008 12:42 pm [phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable
Re: Training an artificial neural network for the micro of marines
If something is lacking from BWAPI, you could either implement it on top of BWAPI or open a github issue. 

- Leckofunny
- Posts: 27
- Joined: Mon Apr 04, 2016 7:41 pm [phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable
Re: Training an artificial neural network for the micro of marines
The attack issue is very well known already.
I've got other goals to achieve, so I'm done with StarCraft for now.
I've got other goals to achieve, so I'm done with StarCraft for now.
-
- Posts: 254
- Joined: Thu Dec 18, 2008 12:42 pm [phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable
Re: Training an artificial neural network for the micro of marines
That's too bad. 
What you were working on looked really promising!

What you were working on looked really promising!
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable
Return to “AI development (BWAPI Bots)”
Who is online
Users browsing this forum: No registered users and 11 guests