Internet engine matches

[Joost Buijs]

One way out is playing faster games, 1 minute is long already, but Bert's 5 minutes is like an eternity.

Matches at 1 to 5 seconds per game are much better in my opinion, and you don't need to change the rules.

Only thing is, that this can be difficult to enforce from a hub standpoint, because of communication and GUI overhead.

I agree that faster games could be a solution, even with games of one second the communication and GUI overhead should not be a problem because it is possible to compensate for that by adding time-stamps to the transferred moves, this is already common practice for computer chess tournaments that run over the internet.

Another concern is time-control, in my experience there are several draughts engines surpassing the allowed time. With chess an engine that surpasses the allowed time will lose immediately, even when it is just 1 microsecond. With draughts the existing GUI's don't seem to look at this at all, this has to be fixed before tournaments with very short time-control become feasible.

[MichelG]

How about a match at 1 second+0.1 second per move? With incremental time control you also don't need to stop at 75 moves, just let the engine run until the 25 move rule forces a draw.

I am up for it. It would take some modifications for me though. Currently the GUI would eat up the thinking time. And the engine requires a bit of a setup for threads, clearing hashes etc, but 100 ms per move should work quickly.

Faster might also be doable 0.1 seconds + 0.01 seconds per move, but that might require more effort reducing the fixed cost per move.

Michel

[Joost Buijs]

I never understood the weird time-control with a fixed time and a fixed number of moves which often produces unknown results. Fisher time-control with increment is the only way to go.

I often let the engine play games of 10 sec. with 0.1 sec. increment, usually that doesn't give problems. The only problem that arises is when the program starts using endgame databases, then it will suddenly take a long time to load and decompress them, only a pre-loaded 6P EGDB (or less) will work with very short time-controls.

When you play a match it's not necessary to clear the hash-table in-between games, usually I do, but I think Kingsrow does not. With ballot matches many positions in these games seem to reappear, so it could be better not to clear the hash-table in-between.

Joost

[Ed Gilbert]

I never understood the weird time-control with a fixed time and a fixed number of moves which often produces unknown results. Fisher time-control with increment is the only way to go.

Joost, I agree completely.

Kingsrow does not clear the hashtable between games. If you use an aging mechanism then probably most of the positions get overwritten by the time they would be useful in the next game, unless you're using a huge hashtable.

I have run many engine matches with 0.1 sec increment and don't see any problem using the 8pc db, as long as it's on an SSD. Kingsrow egdb caching doesn't load new data blocks from disk at leaf nodes, but it still does lookups if the positions are already in cache.

Using 0.1 sec increment with multi-threading doesn't work very well in kingsrow because it uses the YBW threading model, and I think the rapid turnover of split points makes it inefficient for short searches. I'll probably change it to lazy SMP because I think that model should start up quickly with no initial loss of efficiency.

-- Ed

[Joost Buijs]

I have run many engine matches with 0.1 sec increment and don't see any problem using the 8pc db, as long as it's on an SSD. Kingsrow egdb caching doesn't load new data blocks from disk at leaf nodes, but it still does lookups if the positions are already in cache.

Hi Ed,

For the endgame database, I use Windows file mapping. Single-threaded, I see a significant drop in N/s the moment the databases are loaded. The speed drops from about 10 MN/s to 1.5 MN/s. This sometimes lasts for a few seconds, which is long enough to occasionally miss a crucial variation.

Multi-threaded with lazy-SMP, I see almost no drop in speed because the threads are looking at different variations, and the threads usually do not need the endgame database at the same time.

Using 0.1 sec increment with multi-threading doesn't work very well in kingsrow because it uses the YBW threading model, and I think the rapid turnover of split points makes it inefficient for short searches. I'll probably change it to lazy SMP because I think that model should start up quickly with no initial loss of efficiency.

In an older version of the program, I also used YBW. The 'time to depth' of YBW is better, but lazy-SMP makes the search wider, which makes LMR errors occur less frequently.

I keep the threads running continuously and use condition variables to start and stop them. This is very fast; starting a thread this way takes about 0.1 µs.

Joost

[MichelG]

I often let the engine play games of 10 sec. with 0.1 sec. increment, usually that doesn't give problems. The only problem that arises is when the program starts using endgame databases, then it will suddenly take a long time to load and decompress them, only a pre-loaded 6P EGDB (or less) will work with very short time-controls.
Joost

Still that would mean about 18 seconds per game. Currently 60 seconds on 1 thread already is mostly drawn.

If you factor in multithreading and you scale for instance a factor 6 on 16 threads, this corresponds to 108 seconds on a single thread.

Every 10-fold time reduction might increase non-draws only by a factor of 2-3. So timing needs to be a lot faster than this, or alternatively drop multithreading altogether.

[BertTuyt]

I still believe that faster games is not the way forward.
The main way out, but that is my 5 cents, is something like killer draughts, or breakthrough draughts.
With the boundary conditions that practice should show that with increasing depth, the draw ratio does not move to 100% with the same speed as regular draughts.
Long ago I did some experiments with 2 Scan engines going deep with a search depth difference of 4 ply.
And this test revealed that above 20 ply the difference tend to go to (almost) zero.
On my list of things to do is still doing this experiment for both variants, as Scan supports both, and has endgame databases for this purpose.

Bert

[Joost Buijs]

Still that would mean about 18 seconds per game. Currently 60 seconds on 1 thread already is mostly drawn.

If you factor in multithreading and you scale for instance a factor 6 on 16 threads, this corresponds to 108 seconds on a single thread.

Every 10-fold time reduction might increase non-draws only by a factor of 2-3. So timing needs to be a lot faster than this, or alternatively drop multithreading altogether.

I agree, 1 minute single core matches between strong programs on a modern computer will be completely drawn 99% of the time.

For these tests I always use a single thread, for multi-thread I don't use YBW but lazy-SMP, which has a totally different scaling behavior. YBW does not scale very well beyond 8 threads, that's why I switched to lazy-SMP.