Computer tournament and tie break

TAILLE · Post by **TAILLE** » Tue Jun 02, 2009 20:13

Hi Feike,

FeikeBoomstra wrote: I agree, with a tournament it is important that in case of a tie, there is a spectacle.
I also agree, that operator skills should not be the deciding factor.

But why not ignore the operator time. We have to trust the engine's own judgment about the time spent, but to me, that's not a big deal. So only the time the engine is running (in its own time, not the pondering time) is counted for, and the operator can take his time to copy the moves without errors.

Kind regards,

Feike.

I agree with you Feike the operator time is really a worry as soon as we limit drastically the time available.
Let's take for example 5' for all the game (75 moves). That mean that on the average 4" per move.
Now consider 2 operators A and B. A needs 3" per move for operate the move and B needs only 1".
For A the program will spent only 1'15" for its moves and, as a consequence, it will spent 8'45" on the opponent moves.
For B the program will spent 3'45" for its moves and 6'15" on the opponent moves.
As you mentioned Feike, that means that the operator skills becomes a deciding factor.
IMO it is not reasonnable to allow less than 15' per program for a game.

Rein Halbersma · Post by **Rein Halbersma** » Tue Jun 02, 2009 22:28

TAILLE wrote:Hi Feike,

FeikeBoomstra wrote: I agree, with a tournament it is important that in case of a tie, there is a spectacle.
I also agree, that operator skills should not be the deciding factor.

But why not ignore the operator time. We have to trust the engine's own judgment about the time spent, but to me, that's not a big deal. So only the time the engine is running (in its own time, not the pondering time) is counted for, and the operator can take his time to copy the moves without errors.

Kind regards,

Feike.
I agree with you Feike the operator time is really a worry as soon as we limit drastically the time available.
Let's take for example 5' for all the game (75 moves). That mean that on the average 4" per move.
Now consider 2 operators A and B. A needs 3" per move for operate the move and B needs only 1".
For A the program will spent only 1'15" for its moves and, as a consequence, it will spent 8'45" on the opponent moves.
For B the program will spent 3'45" for its moves and 6'15" on the opponent moves.
As you mentioned Feike, that means that the operator skills becomes a deciding factor.
IMO it is not reasonnable to allow less than 15' per program for a game.

Perhaps for decisive games one could play an automated DamExchange match?

Ed Gilbert · Post by **Ed Gilbert** » Tue Jun 02, 2009 22:50

As you mentioned Feike, that means that the operator skills becomes a deciding factor. IMO it is not reasonnable to allow less than 15' per program for a game.

Gerard, I think you misunderstand Feike's suggestion (which I think is very good). I think he is saying, "don't use a table clock, rely on each program's clock to display the correct elapsed time that it has used". That way very fast time controls can be used in a tie break, and operator time does not count as time used by either program. I think this is a good idea. It only means that you must trust each program to correctly track and display the time that it used for its searches.

-- Ed

FeikeBoomstra · Post by **FeikeBoomstra** » Tue Jun 02, 2009 23:01

That's indeed what I mean.

Ed Gilbert · Post by **Ed Gilbert** » Tue Jun 02, 2009 23:11

I am not sure to really understand the meaning of the SB.
Let's take 2 programs classified near the middle of the tournament and let's suppose that these two programs realize each 9 draws and one loss.
Program A lost against the best program of the tournament and program B lost against the weakest program. According to SB there are no hesitation, program B is far better than A. Is it really the common sense ? It is a little strange to decide that it is better lo lose against the weakest program rather than against the strongest one but in the other hand to manage to obtain a draw against the best program may also be a good performance. As far as I am concerned I am really hesitating.

Gerard, you're right, and I am surprised that it works this way. Could it be that SB is designed more for games like chess, where it is much harder to draw against a stronger opponent than in draughts? I'll have to think about this some more.

-- Ed

TAILLE · Post by **TAILLE** » Tue Jun 02, 2009 23:52

I understood Feike suggestion but, for the time being, I have no real opinion on it. That is the reason why I only confimed that the operator skill may be a problem, without answering to the question.
My problem is that this suggestion will change my time handling which is bind to the existence of a real clock :
When Damy has chosen its move it shows the move on the screen but it does not not stop its clock. As an operator I play the move chosen on the board, I stop the official clock and then I have to click on a button to inform Damy that it can now stop its internal clock.
If now we have to play quick games based on the internal clock of each computer you can see that some changes (certainly not very difficult) have to be made.
In addition, while entering the opponent move (click on the original square, click on the destination square and in rare cases choose of the capture) Damy is only waiting without calculation. Should we play quick games with only very few seconds per move that becomes inacceptable and I have also to change this point which is may be not so simple.
But the major point is probably elsewhere :
Suppose that each program have 5' minutes (4" per move) for the game (75 moves) and suppose that each computer used its internal clock as a reference. As a consequence there will be no stress on the operator and they each can take let say 3" per move for operational purpose. That means that in reality each program will have 3" per move to choose its move and 9" per move in calculation during the opponent search and during operations. In that case I suspect I will completly change Damy algorithm in order to have the best use of these 9" available.
It is not that important with 20' for the game but with only 5' it is a little different.
At least it is my opinion.

FeikeBoomstra · Post by **FeikeBoomstra** » Wed Jun 03, 2009 00:17

a) OK, I understand, you inform your engine about one part of the operators handling, so to be the best you do to be can complete, is to also start your clock again at the moment you touch the first piece.

b) most programs have their engines separated from the GUI. (threads, sockets, whatever), so it should be not too complicated to keep pondering during move entry.

c) I just started today with trying to implement pondering, as my guess is that real time efficiency : pondering time efficiency = 4 : 1

TAILLE · Post by **TAILLE** » Wed Jun 03, 2009 10:30

Hi Feike,

FeikeBoomstra wrote:a) OK, I understand, you inform your engine about one part of the operators handling, so to be the best you do to be can complete, is to also start your clock again at the moment you touch the first piece.

b) most programs have their engines separated from the GUI. (threads, sockets, whatever), so it should be not too complicated to keep pondering during move entry.

c) I just started today with trying to implement pondering, as my guess is that real time efficiency : pondering time efficiency = 4 : 1

In the current Damy version the engine is not separated from the GUI. Of course I have in mind to separate them but I was not in a hurry because for games with 20' minutes per program it does not harm at all. It is not too difficult to do but it will take a certain time.

What do you mean by "pondering time efficiency = 4 : 1". Due to my bad enblish I guess I do not see clearly what you call pondering. Is it the calculations you can do during opponent searching? Can you explain a little ?

FeikeBoomstra · Post by **FeikeBoomstra** » Wed Jun 03, 2009 11:42

Hi Gerard,

yes you are right, if the engine is pondering, it is extending the search as long as it is not told to stop. If you do this in the opponents time, the gain is that you preset your hash table.

But you don't know what the opponents move will be, so you start analyzing on the position that was the result of your last move.

If you reach a search depth during pondering of e.g. 13 than the useful part of the hash table is as if the search depth was 12 ply,

So to obtain the same result during pondering, compared to searching while you are really on move, you have to search one extra ply.

I realize now that my 4:1 ratio is not correct, that's about the ratio if you increase your search-depth with 2 plies, with one ply it is then around two to one,
so imo pondering is about half as effective as searching while on move.

I hope I made myself clear, it is difficult to me to describe what I mean.

TAILLE · Post by **TAILLE** » Wed Jun 03, 2009 12:31

FeikeBoomstra wrote:Hi Gerard,

yes you are right, if the engine is pondering, it is extending the search as long as it is not told to stop. If you do this in the opponents time, the gain is that you preset your hash table.

But you don't know what the opponents move will be, so you start analyzing on the position that was the result of your last move.

If you reach a search depth during pondering of e.g. 13 than the useful part of the hash table is as if the search depth was 12 ply,

So to obtain the same result during pondering, compared to searching while you are really on move, you have to search one extra ply.

I realize now that my 4:1 ratio is not correct, that's about the ratio if you increase your search-depth with 2 plies, with one ply it is then around two to one,
so imo pondering is about half as effective as searching while on move.

I hope I made myself clear, it is difficult to me to describe what I mean.

OK Feike I understand.
I have to confess that I am not quite happy with the current pondering of Damy and with this idea to make quick games I will completly change my approach.
My idea is the following. Basically you have more time on pondering than on searching your move, because the time for operations are added to pondering and not to searching. Let's then suppose you have 4" (on average) for searching and 7" (on average) for pondering. As soon as you have definetly chosen your move your program knows what will be the "probably" answer of the opponent. You can then make the assumption that this move will be effectively chosen by your opponent and you can then use these 7" to anticipate your next searching. If the opponent choses effectively this move then you can afford to answer immediatly and save your 4". If the opponent choses another move, then due to all time saved, you can probably take 6 or 7" for searching.
What is your feeling ?

FeikeBoomstra · Post by **FeikeBoomstra** » Wed Jun 03, 2009 19:24

But there are many cases where there are say 2-4 realistic opponents moves. For offline analysis purpose, I have a routine that calculates the move value for all moves in a certain position, and with that routine you see rather often that there are two moves with exact the same value, and/or some moves very close to the best value.

So my idea is, don't take the risk on the best move from your previous run, but just start analyzing the same position as your opponent is doing. Alpha-beta will divide the time spent over the various opponents moves.

I have to go through a paradigm shift myself. I started with an engine as a memoryless entity, just analyze the given position, and return the best move. Now I have to rethink that decision, for the hash contains a lot of valuable information, gathered in the previous run.

Up to now, I use for the next move generation, from the hash only the best move for a position found in the hash. The related best move value I discard, because I don't recalculate the search-depth and distance from root for that position found.

So, a lot of work waiting.

Feike

Ed Gilbert · Post by **Ed Gilbert** » Fri Jun 05, 2009 14:07

Pondering strategies were discussed on the Talk Chess forum a while ago: http://www.talkchess.com/forum/viewtopi ... c87402a054

Rein Halbersma · Post by **Rein Halbersma** » Fri Jun 05, 2009 14:38

Ed Gilbert wrote:Pondering strategies were discussed on the Talk Chess forum a while ago: http://www.talkchess.com/forum/viewtopi ... c87402a054

Yes, but I think the viewpoint of Bob Hyatt is not optimal. I prefer Harald Lüßen's idea (similarly to what Feike does) to simply flip the color and start searching for the best move for your opponent. Hyatt's strategy is to gamble what the best move is (based on your previous search) and then give 100% of your pondering time to finding a best response to that guess. If you guess right, you gain time, but if you guess wrong you have to start searching again. I agree with Feike to let the search divide its time over the relevant moves your opponent can make.