Next Season and finally Champions I (Bot Update)

in #steemmonsters5 years ago (edited)

Another season and my bot finally made it into Champion I. 150 reward cards and ELO of 5277.

Improvement explained

At the start of last season I switched to a regularization approach for my Monte Carlo Tree search. Before that my approach was as follows:

  1. Pick a team through tree traversal
  2. Evaluate this team against n teams and sum up the reward function values
This approach had two issues: with larger n the MCTS could do less iterations because it was using more time on simulations. Plus rewards could cancel each other out and overfitting might take place. Bad design in general.

My new approach:

  1. Pick a team through tree traversal
  2. Evaluate this team against one random team from a pool of n teams
Now the MCTS isn't stuck as much in simulations and can do more iterations. It should also counter overfitting on one specific team in general.

With that change aicu managed to play in the lower Champion I league. Somewhere between rank 90 and 70. Aside from that, I bought the remaining legendary summoners at max level (including Archmage Arius) at the end of this season, which gave aicu a little push as well.

Sure, Aicu is playing now with a 95% max level deck, but how did the changes affect aicu-chan a deck with a couple reward cards and only level 1 Summoners? Well, it pushed all the way to Silver II from Bronze I last season. The corresponding chart is in aicu chan's update post.

Improvements?

Further improvements are starting to get more complicated. For now I'm trying to figure out if I want to follow the "intuition" approach of AlphaZero or try some other things along the lines of temporal difference learning. Need to brainstorm that a bit, the approach of AlphaZero shows great promises in Hearthstone, Go, Chess etc. But I'd actually like to experiment with other approaches before I go down that route. The Trie approach which I explained last time is something I want to explore further.

Giveaway

Naga Fire WizardNaga Fire WizardJavelin ThrowerJavelin ThrowerJavelin Thrower

For this giveaway there's less variety but more Earth and Fire Splinter awesomeness :). Aside from that the same rules apply as last time. Everyone who comments below with a valid SM username get's added to the giveaway pool. Cards are distributed at random at the end.

Sort:  

thanks for the giveaway. username same as steem name i.e. @gillianpearce

No problem, you're welcome :)

Posted using Partiko Android

Champion I with a bot... for that I take my hat off👏 really impressed with that achievement, the lvl1 account does play nice as well. Maybe even more impressive than the big one.

Against what pool of teams are you checking?

Username is @guchtere

Posted using Partiko iOS

Thanks, it took some work to get it there. But I hope it doesn't stop there.

Regarding the team pool. I'm testing multiple strategies right now. Deterministic and random ones. I think right now the one random approach works the best.
30% to 70% from the pool is selected from the opponent's past team selection and the rest are random past teams. Right now the pool has 10 teams each round.

Woah, I'm pretty impressed that you managed to get so far with a bot! It's a interesting puzzle to solve!

My Steem handle is @bengy

Thanks :) Yes, solving that puzzle is really interesting. I just need to get my mindset away from my using MCTS and go to another approach :). Because I don't think I can get much further with my current approach.

Congratulations on making champ 1. I'd love to say I understood your post "tree traversal"!!??!! Lol

Guessing it's something to do with match predictions and "best fit" team on expected outcome?

Anyway, I'm stuck with level 3 & 4 summoners so anything you got to giveaway would be greatly accepted.

Cheers buddy and good luck for this season.

Thanks :) im not sure if tree traversal ai the right terminus there. Usually it means to go through each node of a tree. What I meant is a path from the root node to a leaf node. Where each node on the way represents a team member in order. So B would be Lyanna, D Flesh Golem H Wood Nymph and so on. The tree which my algorithm builds is just a bit deeper and wider. Every possible valid team is stored in it after all. It then simulates a battle between the selected team and one enemy team. The score it gets from that is propagated back up on the path it came from. Through that way the algorithm gets more and more infos which monsters are suitable at position n. So in that case it might learn that lyanna at position 0 and flesh golem at position 1 are a good partial team team, but wood Nymph is not optimal at position 2. And so on. I think I'll do a more informative and better illustrated post on that sometime :)

Posted using Partiko Android

So how do you go about running this and can it be run from a normal PC? I'm guessing the ai would be compatible for all monsters/summoners combo regardless of levels of each.

This programme runs 24/7 and is about finish position rather than DEC collection, because I'm guessing games played would wipe dec% to zero?

I'm not stupid, I've just never found or had the need to learn this stuff. It all makes sense and if I had a spare team, I'd set it up for card collection.

Yes, it can be run on a normal PC. Im doing that right now. My pc is running 24/7 right now. I think I'll switch to a server in the near future. The faster the CPU (plus if I implemented multithreading) the better the estimation gets.

And yes, it's compatible with any gameplay combination. It all depends on the underlying simulation. If it's implemented there correctly, the MCTS works with it. That's the advantage of this approach compared to a statistical one. With a statistical approach you need lots of data and if the devs add more restrictive rulesets the statistical approach will get less and less effective. For now it's still really good. See top 100 for that.

Regarding DEC%, I wiped mine out last season. But this season I'll play a bit slower as I haven't seen a huge advantage in regards of rank. Aicu was running high freq (15 sec between matches) and aicu-chan hourly. There was a more stable gain to aicu-chans ranking. So I set aicu back to hourly as well. And see how it plays out this season.

I don't think you are stupid, you just set your sight on other topics. I sometimes forget that what I've learned is not common knowledge. I really need to consider that when writing posts and explain things better :)

I am really impressed by your bot man! Good job! Heres my handle: @andywong31

Thank you, but it still has a long way to go. I want to see aicu at #1 using traditional reinforcement learning techniques :)

Posted using Partiko Android

very nice, I'm curious to play more against aicu to see how it works out
SM: @schneegecko

I hope so too, because if aicu matches up more with you it plays more in champions league ;)

Great update. Fantastic job!

Thanks :)

Posted using Partiko Android

Thanks for the interesting update as usual.

My SM name is @handtalk5

Posted using Partiko Android

You're welcome, thanks :)

Grats, and thank you for the giveaway chance

You're welcome, and good luck in the giveaway :)

Posted using Partiko Android

Coin Marketplace

STEEM 0.27
TRX 0.13
JST 0.032
BTC 62699.43
ETH 2963.61
USDT 1.00
SBD 3.61