Replace ELO with a Tiered Ranking system
Without rehashing all the ways ELO fails to represent a players skill and it's unsuitability for a game like Catan, other than addicting a player keep trying to get this useless number higher, the only purpose I can see to it would be to rank a player to help setup games, or see someone's general experience. So why not implement something that does directly that, a simple tiered ranking system of some sort.
How about for every 10-20 games someone wins against similarly ranked or higher players, we bump them up a rank. Maybe at the lowest rank, a number of lucky wins will bump you up to the next level, but as you get ranked higher and higher the likelyhood of a lucky win is lower so the rank means something. And unlike ELO, the rank never gets lower because unlucky (or targetted) losses don't relate to your skill as a player, so why should your rank decrease.
Then, when starting games, you have the option of a ranked or unranked game. A ranked game would be against players of similar rank [or if needed maybe one rank up if there aren't enough people online], so you have a chance of getting wins to increase your rank, but after a while it might be harder to find players for a game. And an unranked game would be against any level (maybe a slight priority to similar level to keep it balanced and fun), more people to play against so easier to get a game going, but takes off the significance of a win, so it would hopefully be more fun of a game.
Some implications of this is might be just playing amongst a small group of friends might slow your growth in rank, or after getting a high enough rank, it's harder to find skilled players to play against to increase your rank further; but I think this is OK, it largely should be a fun game, and just playing lower skilled players doesn't mean you are endlessly getting better (which ELO implies). Plus, this would be a good setup for tournaments (a future feature of catan universe?) where players can boost their rank more as this would hopefully draw in larger numbers of skilled players at one time (and tournaments adding another revenue stream for the developers)
Any thoughts/additions/tweaks/alternatives? [I don't remember Catan Online, if this is re-inventing their approach, or how it was handled in the past. Or are there other games that tried this approach and what were it's flaws].
There definitely needs to be a way to decide if you are playing for fun, or want a competitive game against similar players, and having a meaningful number/rank (which ELO currently does not provide)
[unresolved questions in my mind
- ranking when AI comes into the game: I think if the AI beats you, then you didn't win the game. Primarily this is to discourage people from racking up wins by gaming the AI to secure a win [usually by targetting only the human players and allowing the AI to win]
- if someone always comes in 2nd, is there a partial increase in rank? Part of me thinks no, they didn't win, and if they can't ever win then they shouldn't increase in rank -- part of me thinks maybe they should get "1/4 win" or something, so that enough 2nd place finishes boosts them a level, but I think this defeats the purpose of the system, to show who can pull off wins against similarly ranked players... so I'm leaning towards only winning counts ]
[One more unexplored question: How to prevent rank skew (ie, the all the people in the rank are actually similarly skilled)?
I haven't really explored the idea and math around whether there is a potential for the ranks to get skewed early on if the pool of catan players isn't large enough, or by "early players in catan universe" versus people who join in a year or two, or if someone isn't playing enough different players that their wins are meaningful. At this point, I was just hoping that the numbers would trend towards meaningful after a few jumps in rank.]
[And one random last thought for the moment: ranking and ranked/unranked games might also help developers decide which AI personality to add to the game as a replacement to dropped players - a competitive aggressive one, or a generous trading less aggressive one.]