Training an artificial neural network for the micro of marines

Anything related on how to build / improve your AI using BWAPI, BWTA, BWSAL, etc.
User avatar
Leckofunny
Posts: 27
Joined: Mon Apr 04, 2016 7:41 pm

Re: Training an artificial neural network for the micro of marines

Postby Leckofunny » Sun May 15, 2016 8:22 am

I still have to troubleshoot the feedback logic for an attack. I have to capture three times the overall state for my fitness function. One is right before the action execution capturing the initial friendly state. The second record saves the final friendly and the initial enemy state right after launching the projectile or the strike. The last final enemy state is being captured on impact, so as soon as the enemy's hit points decrease. The second capturing point is a huge problem maker right now:

For a projectile to be launched, the unit has to get in range, the unit's weapon has to be not on cooldown and then the attack animation has to be played. The BWAPI doesn't really provide enough information to indicate the launch of a projectile. I can just estimate hard coded on which frame that projectile was launched on. The booleans isStartingAttack and isAttackFrame are useless for this case. isStartingAttack is triggered on the first frame of the attack animation. isAttackFrame is true for each attack animation frame. The projectile is not launched at the end of the completed attack animation. It is getting fired somewhere in between. The whole attack animation of a marine takes 6-8 frames. The projectile is launcher after 3 or 4 frames. Also the cooldown allows the attack animation to play, but to actually launch the projectile as soon as the cooldown equals 0.


I'm not exactly sure when this will be fixed by me, because right now I have to start writing the major parts of my thesis. So far I spent 5 weeks on development. 9 weeks are left, so I gotta get started on the righting with just a little time for troubleshooting.

This paper written by Dave Churchill and Michael Buro describes pretty well the issue of attacks (page 4, Unit Micro-Management).
User avatar
Leckofunny
Posts: 27
Joined: Mon Apr 04, 2016 7:41 pm

Re: Training an artificial neural network for the micro of marines

Postby Leckofunny » Tue May 24, 2016 12:41 pm

I don't really have news for the concern of this thread, but I'm working on a small AI module to analyze attacks of a single unit. So hopefully I gain more insights or even a breakthrough. If that fails, I probably consider a very rough and broad time frame for an attack to occur. In the end, my main goal is to gather not conflicting training data.

Concerning evasive or defensive actions, such as movement, I've got the idea about a somewhat hybrid approach. The current fitness function compares states in order to measure the outcome of attacks. A strict set of rules should measure the other actions like movement. Such a set might be a typical decision tree. Obviously the neural net would learn the behavior of the decision tree, but there is still that dynamic part for attacks.

I might dig deeper into the neural net applications for chess or GO. My supervisor is thinking about evaluating an action after quite some more time, like multiple seconds. Since in chess you can't measure a move right before the next move.
User avatar
warwolf30
Site Admin
Posts: 7
Joined: Mon Apr 04, 2016 12:33 am

Re: Training an artificial neural network for the micro of marines

Postby warwolf30 » Tue May 24, 2016 5:53 pm

Hey, I'm not sure if you already looked into the following links (all of them are in the StarCraft AI wiki), but the following ones may be relevant to you:
User avatar
Leckofunny
Posts: 27
Joined: Mon Apr 04, 2016 7:41 pm

Re: Training an artificial neural network for the micro of marines

Postby Leckofunny » Tue May 24, 2016 6:55 pm

Thanks for the reference. I didn't take a look at these before.
User avatar
Leckofunny
Posts: 27
Joined: Mon Apr 04, 2016 7:41 pm

Re: Training an artificial neural network for the micro of marines

Postby Leckofunny » Tue May 31, 2016 11:14 am

Last week I pretty much confirmed that there is currently no reliable way to work precisely with attacks. This issue pretty much breaks my whole concept/approach. If I just roughly assign some time for attacks, I'll face the problem that I can't separate the impacts done by multiple units.
krasi0
Posts: 254
Joined: Thu Dec 18, 2008 12:42 pm

Re: Training an artificial neural network for the micro of marines

Postby krasi0 » Thu Jun 02, 2016 12:41 pm

From what I've seen, there are bots that seem to handle micro (attacking units included) pretty well even when lan game latency is involved, so perhaps you should dig deeper in some open source bots' code
User avatar
Leckofunny
Posts: 27
Joined: Mon Apr 04, 2016 7:41 pm

Re: Training an artificial neural network for the micro of marines

Postby Leckofunny » Thu Jun 02, 2016 12:47 pm

Right now I'm trying to cover the duration of an attack by just looking at the animation frames of one attack. It won't be frame perfect, but it should reasonable enough to get clean training examples.

I should really look at some bots, thanks for the hint.
User avatar
Leckofunny
Posts: 27
Joined: Mon Apr 04, 2016 7:41 pm

Re: Training an artificial neural network for the micro of marines

Postby Leckofunny » Sun Jun 12, 2016 5:46 pm

For the sake of some result I just composed 120 training examples manually. This is what the result looks like:
https://www.youtube.com/watch?v=13YBM1C4_Ac

Outputs are: AttackWeakest, AttackClosest, RunAway
Inputs are: friendly unit count, enemy unit count, unit hitpoints, over all friendly hit points and overall enemy hit points. The training data can be found here:
https://docs.google.com/spreadsheets/d/ ... edit#gid=0


The approach of generating training data for individual units pretty much failed. I wrote down 3 issues concerning that approach inside my repo's wiki.
https://github.com/MarcoMeter/Starcraft ... generation


So deadline is in about one month. The time for a completely new approach would be highly on risk. I probably could get an extension for one month, but I'm not really convinced for a potential breakthrough.
krasi0
Posts: 254
Joined: Thu Dec 18, 2008 12:42 pm

Re: Training an artificial neural network for the micro of marines

Postby krasi0 » Tue Jun 14, 2016 9:50 pm

There is some progress! I can observe some intelligence emerging the units' behaviour :)
User avatar
Leckofunny
Posts: 27
Joined: Mon Apr 04, 2016 7:41 pm

Re: Training an artificial neural network for the micro of marines

Postby Leckofunny » Wed Jun 15, 2016 6:17 am

There are some news by now. I revealed a few fresh ideas obscured by the fog of war ;)

So the first idea is about treating attacks. I assume now that each attack is successful. There is only a small chance of an attack to miss. That would be the case of an attack being launched towards a dead enemy. Another case has to cover the fact that an attack is most successful if the target gets killed. Fortunately, the BWAPI provides the kill count. This way I should be able to evaluate attacks properly.

In order to add more complexity to my input vector, I'm planning on implementing two radii around the unit. The first one is based on the max attack range and the second one is twice as long. Each area based on that radius will gather independently information about friendly and enemy statistics. This should help the neural net to indicate the level of threat. Just punching in data about enemy health and count is too bold. Like there situations where one enemy is close by and three ones are still out of range. If it was the other way around with 3 foes being close by and one being far away, the input vector would describe both situations completely to be the same. So the two circles and everything what's outside the circle should give much more meaning to the neural net.

The leave of threat could also be used to measure movement actions. Moving back can be considered as good as soon as the threat inside the inner circle decreased.

My "State of the Art" slide show provides some pictures for these ideas: link.

So this is what I'm working for this whole week. If that's some great breakthrough for generating training data, I'll request an extension for my deadline (July 14th). Otherwise I'll craft another more interesting set of training examples and write the thesis.

Return to “AI development (BWAPI Bots)”

Who is online

Users browsing this forum: No registered users and 1 guest