The Age of Rivals AI: A Behind the Scenes Look

One of the things players seem to like about Age of Rivals is the quality of the AI opponent.  A small minority of players have even jumped to the conclusion that the AI must be cheating (it’s not).  I’ve been asked how it works behind the scenes, so I’m dedicating this blog post to that topic.  I will go in-depth on the AI logic and also provide some interesting stats.  

Goal

The initial goal of the AI was to provide a good learning experience for new players, at which point players would transition into playing multiplayer exclusively.  But as we realized that there was a significant number of players who wanted to play single player, I kept improving the AI to play at a higher level.  

During the course of any AoR game, there are basically two kinds of decisions a player must make:

  • Which card to draft out of a possible pool of 3 or 4 cards.
  • Which cards to assign any opponent damage to.

Drafting

To make the first decision (what to draft), the AI scores each card across 17 different categories.  For each category, a function examines the card and the current state of the game and returns a score between 0 and 1.  Then the AI weights each of these results, averages them into an overall score, and drafts the card with the best overall score.  Weights for the categories range from 1 (not very important) to 20 (very important).  

The 17 categories are:

  1. Discounts
    1. How much of a discount am I getting on this card?
  2. Taxes (negative factor)
    1. How much tax do I have to pay my opponent for this card?
  3. Affordability
    1. How affordable is this card given how much Gold I have left and how many cards I still have to buy this round (so I won’t end up with Waste).
  4. Resources
    1. How many new resources will I get with respect to how many resources I and my opponent already have and which round we are in.
  5. Income
    1. How much income will this card generate with respect to how much income I and my opponent already have and which round we are in.
  6. Gold
    1. How much immediate gold will this card give me this round?
  7. Culture
    1. How much culture will this card give me with respect to which round we are in?
  8. Conquests
    1. To what extent does this card allow me to win additional conquests?
  9. Damage
    1. To what extent will this card allow me to make a dent in my opponent’s current armor?
  10. Armor
    1. To what extent will this card help me deal with my opponent’s upcoming attacks?
  11. Field Combos
    1. How well does this card combo with my other cards currently in the field this round?
    2. How well does this card counter my opponent’s cards currently in the field this round?
  12. City Combos
    1. How well does this card combo with other cards I have drafted for my city so far?
    2. How well does this card counter the cards my opponent has drafted for his/her city so far?
  13. Draft Combos
    1. How well does this card combo with the 4 cards my opponent might pass to me next turn?
    2. How well does this card counter the 4 cards my opponent might draft this turn?
  14. Field Counters (negative factor)
    1. Does my opponent have cards in the field right now that counter this card?
  15. City Counters (negative factor)
    1. Does my opponent have cards from previous drafts that counter this card?
  16. Draft Counters (negative factor)
    1. How well do the other 3 cards I must pass to my opponent counter this card?
    2. How well do the 4 cards my opponent might draft counter this card?
  17. Special Abilities (both positive and negative)
    1. Special card-specific weighting to help the AI understand when to use cards with abilities that are not covered by the default combos and counters logic.

The first 10 categories are fairly straightforward.  The AI looks at the current state of the game (how much culture/armor/attack/economy each player has and what round it currently is) and tries to find the card that will best benefit it.  

Abilities

The last 7 categories attempt to handle Abilities, which are trickier.  It is assumed that the cost of each card is fair and accurately reflects its value under ordinary circumstances, so the AI is basically looking for extraordinary circumstances to take advantage of.  If it has a lot of Infantry cards, then Shieldwall is a high value card since that ability represents a combo opportunity with Infantry cards.  

Categories 11-16 (combos and counters) handle most card abilities, and for a long time that seemed okay enough.  But it eventually became obvious that for some cards it’s harder to describe what a combo or counter is, and so category 17 was added to provide custom logic for those cards.  Examples include Plaguebearer, Master Thief and The Oracle.  These are cards that you should only buy under very specific circumstances, and if the AI buys one when it shouldn’t, it can ruin the illusion of intelligence completely.  About 25 cards have such custom logic.

The first version of the AI probably only had about half of these categories.  Whenever I embarked upon a series of AI improvements, I would simply play a game and wait for the AI to do something that seemed obviously sub-optimal, and then I would analyze its logic to figure out why it made that choice.  This usually resulted in a re-write of a category, the addition of a new one, or the adjusting of weights.  

Assigning Damage

Combat damage assignment follows a similar pattern, but with only 10 categories.  The AI randomly assigns damage and then calculates the total value of the cards left over.  It does this 1000 times and then picks the permutation that resulted in the highest total value.  I probably could have done this in a more rigorous way, but in practice it seemed to work pretty well so I never improved it.  

The 10 combat categories are:

  1. Culture
    1. What is the scoring value of the leftover card?
  2. Value
    1. What is the overall value of the leftover card, as defined by its base cost?
  3. Armor
    1. How much armor does the card have leftover after damage has been assigned?
  4. Waste/Ruins (negative factor)
    1. Is this card Waste or Ruins? (Such cards should have been sacrificed typically)
  5. Buffing Culture
    1. Is this card currently buffing the culture value of other cards?
  6. Economic
    1. Is this card currently contributing resources or income?
  7. Post-War Ability
    1. Does this card have a post-war or end-of-round ability that will trigger?
  8. District Combos (same as above)
  9. Deck Combos (same as above)
  10. Special Abilities
    1. Custom logic to account for certain cards that you should or should not sacrifice under very specific circumstances. One example is Shock Troops, which should almost always be sacrificed since it’s going to ruin itself anyways (unless it received a Culture buff). Currently six cards have such custom logic.

Normal vs Hard AI

Over the past year I’ve released several sets of category weightings, including ones that favored more combat, more culture, and everything in-between.  I’ve measured their various win rates and removed poorer performers over time.  There is only one AI out in the wild currently.  

So what’s the difference between the Normal and Hard AIs?  They both use the same set of weights and the same categories of logic.  But the Normal AI just chooses to ignore some of the logic some of the time, randomly.  This is an attempt to simulate a human player occasionally missing something.   

Performance

So how well does the AI actually perform?  Over the last 6 months:

  • The Normal AI wins about 40% of games against all players.
  • The Normal AI wins about 37% of games against players who’ve been playing for at least a week.
  • The Hard AI wins about 61% of games against all players.
  • The Hard AI wins about 54% of games against players who’ve been playing for at least a month.

So both AIs become more “beatable” as players get more experience.  But the Hard AI in particular has the potential to stay fairly competitive.  

But of course there are some players who beat the average and crush the AI 80-90% of the time.  They have probably come up with a play style that is particularly effective against the AI's logic and weights, and over time I can try to improve the AI by analyzing their games in particular.  

Summary

In my opinion, the AI’s main advantages are:

  • The ability to consider all these different aspects of the game. Humans can do this as well though, and the best players do.
  • The ability to remember cards that were drafted in prior rounds that aren’t currently on-screen.
  • The ability to ignore prior strategies and adapt to changing circumstances. Sometimes humans (like myself) try to play out a particular strategy and are too slow to adapt when it’s clear that it’s not going to work out.

The AI’s main disadvantages are:

  • It still doesn’t understand the nuances of every ability perfectly, and it over-simplifies many of them.
  • It makes no attempt to Guarantee cards that work well together, or that are even known to be useful cards. Most humans know over time which cards are better Guarantees, but the AI always chooses these at random.
  • It has no understanding of the current “meta” or which cards are perceived to be better than others. It does not have access to any card win rate data, even its own. Humans remember which cards served them well in the past and can take advantage of that knowledge, but the AI plays every game as if it was its first.

Some next steps towards improving the AI could be to analyze games against players that have high win rates vs. the AI, teaching the AI about more abilities, teaching the AI to be more strategic with the Guaranteed Cards system, and continuing to test alternative sets of weights.  

And I’ll say it one more time.  The AI does NOT cheat in any way whatsoever.  It has no access to extra information.  It does not manipulate the random draft.  It does not know which card you drafted before it makes a decision.  

I hope this has been interesting, and as always I’m happy to answer any questions!

Update: To Grind or not to Grind

The short version of this post is that our newest update (3.8) increases coin rewards in all game modes by 30% and adds coin rewards for losing to all game modes (even Normal AI and Friend games).  Now you can gain access to all the content even faster!  We also fixed some bugs.  Read on if you're curious about why we even have card packs in the game.  

So one of the more controversial aspects of our game (from a player feedback perspective) has been the fact that you have to unlock all the cards and playable characters (rivals) over time.  Some players don't like that a premium boardgame does not come with all the content available upfront.  Others appreciate that the game grows more complex and varied over time and that there are long-term goals to work towards.  

Even when I was debating whether or not Age of Rivals should be a premium or free-to-play game, I was reluctant to have all the content available from day one.  I knew from early play-testing that that would have been overwhelming, and that the starter deck of 71 cards made for a good game.  And I thought it would be fun for the game to evolve over time, encouraging players to adapt to the changing deck and to try new strategies as they gained new cards.  Also, I had too much fun watching my 5 and 7 year old daughters opening cards packs.  "Daddy, I unlocked Arsenia!".  It was (and still is) too priceless.  

Most games (premium or not) do this.  As you progress, you gain new skills and and weapons and encounter new challenges.  The metaphor varies from genre to genre.  Sometimes it's leveling up, sometimes it's getting to a new world, sometimes you earn a new seed packet to counter a new type of zombie.  And the paces varies as well.  Sometimes it's steady and linear, and sometimes it's fast at first but then slows down to something more tedious.

And that's where Age of Rivals runs into problems.  Our metaphor is the dreaded card pack.  And it's normal to assume that card packs equal a slow and tedious grind.  Many CCGs these days are free-to-play with card packs, and it takes forever to unlock everything.  

Our original metaphor long ago was a linear progression of unlocking cards one by one as you won games.  But we switched to the more easily recognizable card pack system so we could release a free-to-play version of the game on Kongregate.  And when we switched back to premium for Steam/iOS/Android, we decided to keep it.  (The whole premium vs free-to-play thing can be a future blog post).

Maybe this was a bad idea, but by then it was fully developed and we didn't really want to redo it.  The important thing was that we changed the pace of content unlocking to linear by eliminating redundant drops.  You never get a card you don't need, so you're unlocking content as quickly on day 30 as you did on day 1.  

But while the pace was pleasingly linear, it was definitely still too slow overall.  We still had one foot in free-to-play-economy land.  Our early Steam players gave us some great feedback, and so we quickly doubled the coin rewards, added rewards for more game modes, and added unlimited missions.  

Today we're going the rest of the way.  We're speeding up the pace again by increasing coin rewards in all modes and (more importantly) adding rewards for loss in all modes.  Everyone should now be able to earn all the Level 3 Rivals and all cards after a pretty reasonable number of games.

Hope you all like the new update.  As always, keep the feedback coming!  

Age of Rivals Goes Mobile

Age of Rivals launches on Android and iOS on Thursday, June 22!

blogMobileLaunch.png

Just like the Dragoons of old, Age of Rivals is going mobile.  We finally have a firm date for our mobile/tablet launches: June 22.  We've been quietly beta testing for the past few weeks to make sure everything looks stable, and now we're ready to go!

Launch Sale

In celebration of these launches, we've reached out to our Moneylenders and convinced them that for the first week the game will be on sale for $.99.  Make sure to grab it and take advantage of this launch discount!

Includes All Content

Just like the Steam edition, these mobile versions will be premium versions of the game with no in-app purchases.  This means that all content unlocks at a linear pace, and you never get duplicate cards you don't need.  As long as you keep playing, you will regularly receive new cards and rivals until you unlock everything.  Remember that as long as you play vs. Anyone or vs. the Hard computer, you earn coins even when you lose the game.  That is certainly a Good (Trade) Deal!  (Sorry, had to do it!) 

We hope you enjoy being able to play from the convenience of your phone or tablet!  We'll post the links to the stores on Thursday when the game goes live.  Thanks!

 

Update: Balance Changes and More

Today's medium-sized update aims to address a few player-requested issues before we officially launch on mobile.  This also includes our second round of card balance tweaks.  We analyzed game data and read all the comments on both Kongregate and Steam forums before deciding which cards to change.  

The biggest source of controversy recently has been Graverobber Baron.  We agonized over this one, but ultimately we decided to leave it unchanged.  There were good arguments on both sides, but we were more swayed by the players who wanted to keep it as-is.  We think this card plays an interesting and somewhat unique role in the overall composition of the game.  The game benefits from having dramatic late game cards that can turn the tide in big ways, and GB is not really one of the more OP ones.  It has almost exactly a 50% win rate, which is lower than several other cards.  But we still might change our minds in the future, so keep letting us know what you think.  

Balance changes:

  • Warmonger -1 attack
  • Silencers -1 culture
  • Patrons -1 gold cost
  • Arms Dealers -1 armor, +1 attack, -1 gold cost
  • Brilliant Tactic -1 gold cost (now free)
  • Blind Bolt Throwers -1 base attack (ability unchanged)

Other changes requested by players:

  • Friend games no longer have a turn timer.
  • You will now hear an audio warning during timed games when you only have 5 seconds left.  This will only happen if you have Sound Effects turned on.
  • The Normal AI is now a little easier.  The Hard AI is still the same.  

We hope you enjoy these changes!  Our next big update will be to get this game onto mobile finally.  

Update: UX Improvements

Today's update brings more player-requested improvements to the game, mostly involving tooltips.  

1) We now highlight buffed stats in green and debuffed stats in red. If you mouseover these stats, the TOOLTIPS will show you where the buffs/debuffs came from and whether or not they are temporary. This should make it much easier to figure out why a stat changed.

2) The tooltip for a card's gold cost will now show you what the original cost of the card was before it got discounted.

3) The tooltip for resources will now tell you which type of card that resource is most helpful for.

These improvements should make it even easier to learn the game and the relationships between the various cards and elements.  This is live on both Kongregate and Steam.  Thanks!

Age of Rivals Launches on Steam March 14

It's been a long road, but we're finally on the verge of bringing Age of Rivals to a second platform: Steam!  The game officially made it through Greenlight on February 16, and we've been preparing for launch ever since.  

The release date is March 14 (Pi Day!).  You can check out our Store Page now and add the game to your Wishlist.  You can also check out our Community Hub.

We will be having an initial launch sale of 10% off.  And as we mentioned before, if you've been playing on Kongregate, you may be eligible for a free Steam copy and we can also copy your account over if you want.  Details here.

This is good news for all our players since the game will support cross-platform play.  We're excited to welcome a new group of players to the Age of Rivals community!  

Age of Rivals on Steam Greenlight and New Trailer

Help Age of Rivals get on Steam by voting for the game to get through Greenlight

The Steam edition of the game is a little different than the Kongregate version:

  • No in-app purchases in this edition. All cards and characters unlock at a fast pace just by playing the game. 
  • Challenge yourself with bonus DLC single-player content (pre-included in this edition)

(If you have spent $10 or more in the Kongregate version of the game, please email support@robotogames.com to receive a free copy of the Steam version when it launches.) Thanks in advance for your help!

New Trailer

Learn how to run a great civilization in 10 easy steps with our new trailer. 

Age of Rivals Featured on Kongregate

Age of Rivals was featured on the front page in the Hot New Games section for the past 2 weeks. We are so thankful to our Kongregate users who have responded very positively to the game and rated it a 4.2 out of 5 with over 100,000 plays. It's been wonderful seeing so many people in the chat and so much activity in the forums.

Three Guaranteed Cards are Greater Than One

Today's update brings some small but often requested changes to the game.  

Screen Shot 2017-01-12 at 11.33.19 AM.png

One of the more common feature requests in Age of Rivals is for slightly more control over strategy by way of more deck-building.  However we also often hear that players like that there is less deck-building in this game, and that they would prefer it stay this way.  

We are also aware that most players don't even get to fully try out the current level of deck-building, because most players never get to the point where they can Guarantee more than one card.  Playing with 3 Guaranteed cards feels pretty different than playing with just 1 (we think).

So before we make any changes, we want to allow more players to play the game as intended.  To that end, we have reduced the number of Rival Cards you have to get to unlock the 2nd and 3rd slots on every rival.  It used to be 2/10/25, and now it is 2/5/10.  This means that many of you will likely have more slots unlocked the next time you load up the game.

Another frequent request is to be able to earn coins by playing the computer.  So we have also added a reward for beating the computer, including Easy, Hard and Campaign games.  It's not as much as for beating a human (and you get nothing for losing), but it's something.

Finally, we have removed the intro Friend Mission, because many players have a hard time completing it.  If you already have it in your Mission List, it will be dismissible.  

Thanks for all the feedback!  We hope these changes improve everyone's experience.  We have also been working on bringing Age of Rivals to more platforms.  More news on that soon!