Training an artificial neural network for the micro of marines

[Leckofunny]

I still have to troubleshoot the feedback logic for an attack. I have to capture three times the overall state for my fitness function. One is right before the action execution capturing the initial friendly state. The second record saves the final friendly and the initial enemy state right after launching the projectile or the strike. The last final enemy state is being captured on impact, so as soon as the enemy's hit points decrease. The second capturing point is a huge problem maker right now:

For a projectile to be launched, the unit has to get in range, the unit's weapon has to be not on cooldown and then the attack animation has to be played. The BWAPI doesn't really provide enough information to indicate the launch of a projectile. I can just estimate hard coded on which frame that projectile was launched on. The booleans isStartingAttack and isAttackFrame are useless for this case. isStartingAttack is triggered on the first frame of the attack animation. isAttackFrame is true for each attack animation frame. The projectile is not launched at the end of the completed attack animation. It is getting fired somewhere in between. The whole attack animation of a marine takes 6-8 frames. The projectile is launcher after 3 or 4 frames. Also the cooldown allows the attack animation to play, but to actually launch the projectile as soon as the cooldown equals 0.


I'm not exactly sure when this will be fixed by me, because right now I have to start writing the major parts of my thesis. So far I spent 5 weeks on development. 9 weeks are left, so I gotta get started on the righting with just a little time for troubleshooting.

This paper written by Dave Churchill and Michael Buro describes pretty well the issue of attacks (page 4, Unit Micro-Management).

[Leckofunny]

I don't really have news for the concern of this thread, but I'm working on a small AI module to analyze attacks of a single unit. So hopefully I gain more insights or even a breakthrough. If that fails, I probably consider a very rough and broad time frame for an attack to occur. In the end, my main goal is to gather not conflicting training data.

Concerning evasive or defensive actions, such as movement, I've got the idea about a somewhat hybrid approach. The current fitness function compares states in order to measure the outcome of attacks. A strict set of rules should measure the other actions like movement. Such a set might be a typical decision tree. Obviously the neural net would learn the behavior of the decision tree, but there is still that dynamic part for attacks.

I might dig deeper into the neural net applications for chess or GO. My supervisor is thinking about evaluating an action after quite some more time, like multiple seconds. Since in chess you can't measure a move right before the next move.

[warwolf30]

Hey, I'm not sure if you already looked into the following links (all of them are in the StarCraft AI wiki), but the following ones may be relevant to you:

• Attack Animation Frame Data extracted by David Churchill
• Previous work doing Transfer Learning for micro
• Custom maps for benchmarking different scenarios. Sometimes the custom maps used for training are wrongly created. These ones ensure a standard way to compare methods and a well defined metrics that can be used as a reward. They have been used previously to train and test a Q-learning for micro

[Leckofunny]

Thanks for the reference. I didn't take a look at these before.

[Leckofunny]

Last week I pretty much confirmed that there is currently no reliable way to work precisely with attacks. This issue pretty much breaks my whole concept/approach. If I just roughly assign some time for attacks, I'll face the problem that I can't separate the impacts done by multiple units.

[krasi0]

From what I've seen, there are bots that seem to handle micro (attacking units included) pretty well even when lan game latency is involved, so perhaps you should dig deeper in some open source bots' code

[Leckofunny]

Right now I'm trying to cover the duration of an attack by just looking at the animation frames of one attack. It won't be frame perfect, but it should reasonable enough to get clean training examples.

I should really look at some bots, thanks for the hint.

[Leckofunny]

For the sake of some result I just composed 120 training examples manually. This is what the result looks like:
https://www.youtube.com/watch?v=13YBM1C4_Ac

Outputs are: AttackWeakest, AttackClosest, RunAway
Inputs are: friendly unit count, enemy unit count, unit hitpoints, over all friendly hit points and overall enemy hit points. The training data can be found here:
https://docs.google.com/spreadsheets/d/ ... edit#gid=0


The approach of generating training data for individual units pretty much failed. I wrote down 3 issues concerning that approach inside my repo's wiki.
https://github.com/MarcoMeter/Starcraft ... generation


So deadline is in about one month. The time for a completely new approach would be highly on risk. I probably could get an extension for one month, but I'm not really convinced for a potential breakthrough.

[krasi0]

There is some progress! I can observe some intelligence emerging the units' behaviour

[Leckofunny]

There are some news by now. I revealed a few fresh ideas obscured by the fog of war

So the first idea is about treating attacks. I assume now that each attack is successful. There is only a small chance of an attack to miss. That would be the case of an attack being launched towards a dead enemy. Another case has to cover the fact that an attack is most successful if the target gets killed. Fortunately, the BWAPI provides the kill count. This way I should be able to evaluate attacks properly.

In order to add more complexity to my input vector, I'm planning on implementing two radii around the unit. The first one is based on the max attack range and the second one is twice as long. Each area based on that radius will gather independently information about friendly and enemy statistics. This should help the neural net to indicate the level of threat. Just punching in data about enemy health and count is too bold. Like there situations where one enemy is close by and three ones are still out of range. If it was the other way around with 3 foes being close by and one being far away, the input vector would describe both situations completely to be the same. So the two circles and everything what's outside the circle should give much more meaning to the neural net.

The leave of threat could also be used to measure movement actions. Moving back can be considered as good as soon as the threat inside the inner circle decreased.

My "State of the Art" slide show provides some pictures for these ideas: link.

So this is what I'm working for this whole week. If that's some great breakthrough for generating training data, I'll request an extension for my deadline (July 14th). Otherwise I'll craft another more interesting set of training examples and write the thesis.