Game Analysis and Player Rating

[Walter]

Most programs currently have various options to analyse a game. It is for instance possible to flag moves that differ more than 0.X from the best move. It is possible to analyse all moves in a particular position. Etc.

I would like opinions though on another way to analyse games and rate players. The ultimate objective is to be enable to rate the strength of players based on their playing level compared to 'optimal play' instead of on how they performed in games against other people (the usual rating system). The hope is that such a rating system could then be used to detect fraud in online or live games.

The idea is to have a program simultaneously (i.e. using 2 cores) calculates the score of the best move and the score of the played move. The best move will of course always be greater or equal to the played move. The game score of each player (black and white) is calculated as the sum of (best move - played move) divided by the number of moves. Hence, the lower the score the better the player performed. The rating of a player is the average of all game scores over time. (if necessary scores could be normalised somehow to make them more presentable).

If in a particular game a player scores significantly higher, then his rating this could be an indication for fraud. If the rating of various computer programs would be known, then you could maybe even have an indication of which program was used.

The idea is of course based on the assumption that the best move calculated by the program is the 'optimal play'. I believe that given today's strength of programs like Kingsrow, Dragon, Damage combined with extensive opening books and end game databases that is a reasonable assumption.

Of course to get comparable results you would need to use identical settings (same program, same analysis time per move, same databases).

A problem I can see would be that the scores programs use to denote a winning move (e.g. from egdb) will dominate the total score. On a typical move the difference between best move and played move will be well below 1 (or 100). But for instance Kingsrow uses 4000 for a (database) win.

I would be interested to learn what other problems you foresee and if there is an interest to try to work out the details of such a new feature.

[BertTuyt]

Walter, I can put it on my things to do list. First priority is however to debug all the features in the Damage GUI I had implemented last year.
Below is a screenshot I made from the game Boomstra - de Voogd (EK 2014) which was analyzed by Ton Sijbrands this weekend.

Horizon 4.2 finds the same mistakes at black move 41 and 44. You can see that as the moves in the moves list are red. Below the move list the analysis. And new is the score graph.

As this was done on my notebook (with limited resolution) the graph is unfortunately rather small.

Bert

[Walter]

Bert,

Thanks for adding the idea to your to do list! Your analysis example is also nice to explore a bit. It shows very little difference between the moves played by Boomstra and Damage. So in my proposal the game score for Boomstra will be close to zero. De Voogd, however, played two moves that were quite far from optimal. Damage shows a different of -0.57 versus -0.12 for the first mistake. That would add 0.45 (or maybe 450) to the game score for De Voogd. The second mistake probably adds something a bit more (as it is the more decisive mistake). So the final game score for De Voogd would be say (450 + 550) / 49 moves = 20.4, clearly an indication that Boomstra played better than De Voogd.

I can also see some more issues now:

- Dividing the score by the number of moves played would mean the game score improves if you keep playing as long as possible even in a lost position.

- The sum of the mistakes might be too similar.

So whether over time the average of the game scores of a player is a good measure to for the strength of players remains a big question though. Anyway, it would be really nice to get the move scores data out of Damage in orde to experiment with various ways of calculating the overall game scores and player ratings.

Regards,
Walter

[Walter]

A machine learning based classfication of a game as played by a human or a computer would benefit from additional features. For instance, you could ouput the piece counts and whether the (significant) difference between the played move and the best move occurs because the player allowed a shot for the opponent or because the player did not take a shot.

The intuition is that 1) humans are far more likely to miss shots than computer programs, and 2) humans are far more likely to make mistakes in the opening or the end game than a computer program with an opening book and egdb.

So the analysis output could consist of [PlayerToMove, ScorePlayedMove, ScoreBestMove, WhitePieces, BlackPieces, ShotAllowed, ShotNotExecuted]