What do you mean by "evaluation" ?

[TAILLE]

Hi,

Is somebody interested by this rather strange question (in the subject) ?


black to play
What is you evaluation of the position ?

Suppose that a GMI has the white side and a beginner has the black side. The very first goal of the GMI in this game was indeed to verify that is pupil has understood one of the more common combination of the game!
In this context white hope that black will choose 23-29 but, at the same time, because this GMI is facing a beginner, he knows perfectly that is is taking no risks at all and, even with 2 men down, he surely evaluates is chance of winning to a number very near from 100% let's say 99,99%.
The evaluation of the position, for white point of view is here really a very high number.

Fisrt conclusion : the evaluation depends on the strength difference between the two players

Now let's suppose two players of the same strength and let's take the 2 following positions :



What is your feeling about this 2 positions. Do they have the same evaluation ?
This is an important point to understand. One of the difficulty of draughts game is that we can have 3 possible results (win, draw or loss). In GO the result of a game is "normaly" win or loss.
As a consequence it is not easy to evaluate a given position with only one number. Why then not trying to use 2 numbers (3 numbers but with a sum equal to 100) ?

In the first position (20 men against 20 men) you may have the following evaluation :
15% win, 70% draw, 15%loss

In the second position (10 men against 10 men) you may have the following (different) evaluation :
5% win, 90% draw, 5%loss

Of course it is quite different but which is the best ? In my view the choice has to be made taking into account a "profil" for the two players.
The profil of a player is its goal, and its goal is somewhere between the 2 extremes :
profil 1 : to maximise the chance to obtain a win
profil 2 : to minimise the chance to concede a loss
The profil of a player could be then a number equal eg to 0 for profil 1, to 100 for profil 2 and to a number between 0 and 100 for all other profils.

Depending on your profil you may prefer position 1 or 2 and that means that the 2 positions must not have the same value.

Example : a strong player hoping to win a major tournament will certainly, at the beginning of a tournament, adopt a profil near profil 1. But in the last round, if the same player need only a draw in order to win the tournament, he would normally prefer to adopt a profil near profil 2.

Complementary question : ignoring the difficulty for the corresponding algorithm, do you think that it could be interesting to have :
"evaluation for white point of view" + "evaluation for black point of view != 0

In the initial position, 2 players having the profil 1 could be both rather happy, couldn't they ?

What is you feeling ? Do you think that it could be interesting to take into account such profils in order to calculate the evaluation of a position ?

I did not try such approach in Damy but it looks like something fun for the future.

[wellnesswrotter]

Maybe, but maybe, it is usefull to know for a computer what the chance of a draw is on one position vs another. Because than the computer could play not the best move, but the most chancefull.

You can use Profile Evaluation in a game against a human player, so that he can make more mistakes.

But Objective Evaluation will prove better in GMI games, and computer-computer games.

[TAILLE]

But Objective Evaluation will prove better in GMI games, and computer-computer games.

I have some difficulties to understand what you mean exactly by "Objective Evaluation" ?

To simplify, let's suppose you have to players of the same strength with a kind of "standard" profil between profil 1 and profil 2.

For a given position, and forgetting the difficulty to determine the evaluation, I can clearly understand the meaning of a "2 number" evaluation like :
30% win, 50%draw, 20%loss
but what is the meaning of what you call an "Objective Evaluation".
I guess it is some mixture of the probality of winning and the probability to avoid losing but it is not clear for me what do you really try to measure with this.
By the way I did the same for Damy but I do not really know what I tried to measure!!! It seems highly subjective; it represents in my mind a kind of advantage but not an"Objective Evaluation" like a probability.

My basic question is the following :

What do you try to measure when building your evaluation ?

[Rein Halbersma]

But Objective Evaluation will prove better in GMI games, and computer-computer games.

I have some difficulties to understand what you mean exactly by "Objective Evaluation" ?

To simplify, let's suppose you have to players of the same strength with a kind of "standard" profil between profil 1 and profil 2.

For a given position, and forgetting the difficulty to determine the evaluation, I can clearly understand the meaning of a "2 number" evaluation like :
30% win, 50%draw, 20%loss
but what is the meaning of what you call an "Objective Evaluation".
I guess it is some mixture of the probality of winning and the probability to avoid losing but it is not clear for me what do you really try to measure with this.
By the way I did the same for Damy but I do not really know what I tried to measure!!! It seems highly subjective; it represents in my mind a kind of advantage but not an"Objective Evaluation" like a probability.

My basic question is the following :

What do you try to measure when building your evaluation ?

Gerard,

A different but related question: how do you tune your opening book for Damy? Do you use TurboDambse, hand-crafted or an automated book (e.g. by dropout-expansion)? It seems to me that grandmasters are really careful in the opening not to give weaker opponents the chance the simplify.

Rein

[TAILLE]

Hi Rein,

Gerard,

A different but related question: how do you tune your opening book for Damy? Do you use TurboDambse, hand-crafted or an automated book (e.g. by dropout-expansion)? It seems to me that grandmasters are really careful in the opening not to give weaker opponents the chance the simplify.

Rein

I know that Damy opening book is certainly one of Damy weaknesses. I build it manually some years ago with the help of turbodambase. I certainly have to spend some time to improve it.

[64_bit_checkers_engine]

What you are talking about is called "playing against a fallible opponent." It has been the topic of several artificial intelligence papers.

The way most a.i. researches handled their experiments was to design two different evaluation functions.

1. Normal Mode evaluation (NM)
2. "Fallible Opponent" evaluation (FO)

When using the NM, nothing is changed, of course.

When using the FO, the "alpha" side of the evaluation function (the side the computer is playing) calls the NM evaluation. When the "beta" side moves, the FO evaluation is called.

This produces two lines of play. If the move at the root is the same, it is played 100% of the time.

If the move is different, a special procedure is called to evaluate "risk."

If the "risk" is too high, the FO line of play is abandoned, and the NM move is played. If the risk is acceptable, the FO move is made.

What was learned from all of this was: it is hard to create an FO evaluation function! Programmers had to create many test positions and invest much time to come up with something that was only being put to good use about 30% of the time.

I have no idea how well these ideas would translate to International Draughts, but you can use this model and see what happens.

[TAILLE]

Hi,

What you are talking about is called "playing against a fallible opponent." It has been the topic of several artificial intelligence papers.

I do not believe that I am talking about "playing against a faillible opponent". For me "playing against a faillible opponent" means that you are normaly stronger than your opponent and you take into account a probability for the opponent to choose a wrong move.
My appoach is different. I do not consider that the opponent is weaker (it may be weaker but is only a particular case) and I do not take into account any probablity for the opponent to choose a wrong move.
I only consider that the objectives of the 2 players are not necessary in opposition.
3 exemples :
1) If player 1 is only interested by a win and player 2 is only interested by not losing then the 2 objectives are in opposition. That means that if a position is good for player 1 then it is bad for player 2. In this kind of game player 1 will take a lot of risks and player 2 will try to simply the game
2) If both players are only interested by a win then the game will be certainly very complicated with a lot of risks from both players. If the result is finally a draw then each player will feel it was a failure
3) If both players are only interested by not losing, the game will certainly be not very interested and, with a draw, each player will be happy.

My feeling is that the best move in a given position may depend not only of your own profil but also on the profil of your opponent.

Note : as you can see, if the objectives of the 2 players are not in opposition, that means that the game becomes a non-null-sum game and new algorithms have then to be used.