Internet engine matches

[jj]

But switching off pondering, and hence the search threads, alone does not solve it.
There can also be memory conflicts because of the memory used by the two programs for the transposition table and endgame database;
Windows will then start swapping memory blocks from RAM to- and from the disk.

Are these blocks really swapped to/from disk? Even if there is plenty of RAM and everything fits into that? Or is it just swapping to/from RAM cache?
Anyway, I think we can assume that both programs are equally affected by this at the start of their respective searches.

Besides the search threads, there can also be thread(s) to load the endgame database into memory, especially for 8-piece database.
These threads also need to be postponed when it is not the program's turn to do a move,
otherwise it will interfere with the other program.

Programs such as Scan and Maximus (and Sjende Blyn, I presume) use a 6-piece egdb which is loaded entirely into RAM at program startup. So there will be no threads running that load egdbs. Kingsrow can be set to use only 6 pieces as well. I found there is very little difference in playing strength between Kingsrow 6-piece and 8-piece, maybe Ed can confirm this.

Everything need to be set/implement such that it prevents interference of the two programs.
That's why I proposed to Krzysztof to use two AMD Ryzen, as in the tournament, seemed to be the most simple.

As it turns out this is not as simple as running on a single computer, for Krzysztof.
Running fair matches on a single computer never seemed a problem before in computer chess (see TCEC) and computer draughts. What do other programmers think about this?

Another matter is time usage. As far as I have observed, programs such as Scan, Kingsrow and Maximus do not use more time than is agreed. Several other programs do use more time than agreed, even showing this on their own clocks, especially when they play against strong(er) opponents and get into difficult positions. It is up to each programmer to implement this correctly, and so far this is not checked automatically.

Jan-Jaap

[Krzysztof Grzelak]

Krzysztof, does this mean you have it working now?

Yes. The Maximus - Kingsrow test match is in progress.

[Krzysztof Grzelak]

As it turns out this is not as simple as running on a single computer, for Krzysztof.
Running fair matches on a single computer never seemed a problem before in computer chess (see TCEC) and computer draughts. What do other programmers think about this?

Such a match will always be unfair. Imagine playing Sjende Blyn - Kingsrow. Sjende Blyn uses 16 cores, Kingsrow uses 8 threads. There is already injustice at this point. Unfortunately, this cannot be changed in the program Sjende Blyn. It is going to change in Maximus, Scan, Kingsrow, Dragon. Of course, the game is on one computer.

[Krzysztof Grzelak]

Test match (not the whole)on two computers Maximus - Kingsrow.
Result - isn't important.

[Ed Gilbert]

As it turns out this is not as simple as running on a single computer, for Krzysztof.
Running fair matches on a single computer never seemed a problem before in computer chess (see TCEC) and computer draughts. What do other programmers think about this?

I've been running matches on a single computer for years. Not only single matches, but many simultaneous matches. It works fine. To require two identical computers is crazy.

The problem appears to be that sjende blyn does not have a user setting to control pondering, or number of search threads. These are very basic controls.

Besides the search threads, there can also be thread(s) to load the endgame database into memory, especially for 8-piece database.
These threads also need to be postponed when it is not the program's turn to do a move,
otherwise it will interfere with the other program.

Kingsrow fills it's egdb cache when the first search is started. This takes about a minute, then it is finished, and after that kingsrow will never use any cpu cycles with pondering off when it is not its turn to move. Simply wait until kingsrow has finished initializing the egdb before starting a match.

[Jelle Wiersma]

May be I should remind you guys that Sjende Blyn was set
to play in a tournament,
against different programs of varying strength,
where each program runs on a seperate computer.
It was not set to play blitz matches, against one program with excellent strength, on one computer.
You asked Krzysztof to play matches with this program, he is allowed to do that if he wants, but as mentioned before the program was not set for matches.

Jelle

[Jelle Wiersma]

Actually this means that it:
Supports to play one game, either manually or via DXP.
Takes risks to win a game against a weaker program / human. The drawback is that it will do the same against a top program,
which increases the chance to loose against a top program.
Utilizes the available processor and memory.

[Krzysztof Grzelak]

Dziś będę pisał polsku. Tak ostatnio często myślałem o warcabach międzynarodowych i zastanawiałem się na grą silników warcabowych na jednym komputerze i na dwóch komputerach osobno. Większości programistów gra na jednym komputerze. Dlatego postanowiłem zrobić taki krótki test dwóch programów warcabowych : Kingsrow i Scan. Mecz jest rozgrywany na dwóch komputerach osobno. Ustawienie programu Kingsrow 1.60b:

Kingsrow 1.60b x64

Opening Book = Best Moves
HashTable Size = 512 MB
DB cache Size = 30000 MB
The base ends = 8 Pieces
Threads = 8
Time = 1 Min / 75 Moves

Ustawienie program Scan 3.1.

Scan 3.1 x64

Variant = normal
Book = true
Book-ply = 4
Book-margin = 4
Threads = 16
Tt-size = 26
Bb-size = 6
Time = 1 Min / 75 Moves

Tak się zastanawiam czy który z programów przegra. W pierwszy meczu są wszystkie remisy - poniżej plik dxp pierwszego meczu. Już wkrótce kolejne mecze i rezultaty. Wszystkie mecze są rozgrywane na następującym sprzęcie:

Processor - AMD Ryzen Threadripper 1950X
Hard disc - SSD Samsung 860 Pro 1 TB
Memory of frames - 128 GB DDR4 2400
System - Windows 10 64 bit Pro

Kolejny mecz zakończony. Tym razem wygrał program Scan 3-1, resztę gier zakończyło się remisem. Następny mecz zakończony zwycięstwem programu Scan. Scan wygrał 4 gry, pozostałe gry zakończyły się remisem. Dziś kolejny dzień testów programów Kingsrow i Scan. Dzisiejszy mecz zakończył się zwycięstwem programu Scan - wygrał 3 gry, resztę gier zakończyło się remisami.

[jj]

Krzysztof, what is your point?

[Krzysztof Grzelak]

Krzysztof, what is your point?

I am doing a two-day test on Kingsrow - Scan on separate computers. I write only in Polish. This time the match ended in a draw. No program has lost or won.

[Ed Gilbert]

I am doing a two-day test on Kingsrow - Scan on separate computers. I write only in Polish. This time the match ended in a draw. No program has lost or won.

This only shows that 158 games is not nearly enough to learn which program is stronger when the programs are not very far apart. Thousands of games are needed. And these are fast 1-minute games. If you play longer games, you get fewer decisive games, so you need even more.

-- Ed

[Krzysztof Grzelak]

This only shows that 158 games is not nearly enough to learn which program is stronger when the programs are not very far apart. Thousands of games are needed. And these are fast 1-minute games. If you play longer games, you get fewer decisive games, so you need even more.

-- Ed

Of course you are right Ed.

[Krzysztof Grzelak]

Match SJENDE BLYN - MAXIMUS (2-move ballots)

Sjende Blyn (2019aug31) vs. Maximus 2.01 1 wins, 0 losses, 156 draws, 1 unknowns

The match was played on separate computers with the same parameters:

Processor - AMD Ryzen Threadripper 1950X
Hard disc - SSD Samsung 860 Pro 1 TB
Memory of frames - 128 GB DDR4 2400
System - Windows 10 64 bit Pro

Sjende Blyn (2019aug31)

Cores - 16
Opening book - On
Strength - Maximum
Pondering - On
The base ends - 6
Time - 15 Min / 75 Moves

Maximus 2.01

Opening book - Best moves
Transposition Table - 32 M positions (25 bits)
Threads - 16
Ponder threads - 16
Pondering - On
The base ends - 6
Time - 15 Min / 75 Moves

[Krzysztof Grzelak]

Match KINGSROW - SCAN (2-move ballots)

Kingsrow 1.61 vs. Scan 3.1 1 wins, 0 losses, 157 draws, 0 unknowns

Kingsrow 1.61 x64

Opening Book = Best Moves
HashTable Size = 512 MB
DB cache Size = 30000 MB
The base ends = 8 Pieces
Threads = 8
Time = 1 Min / 75 Moves

Scan 3.1 x64

Variant = normal
Book = true
Book-ply = 4
Book-margin = 4
Threads = 8
Tt-size = 26
Bb-size = 6
Time = 1 Min / 75 Moves

Match played on a computer with the equipment.

Processor - AMD Ryzen Threadripper 1950X
Hard disc - Samsung 860 Pro SSD 1 TB
Memory of frames - 128 GB DDR4 2400
System - Windows 10 64 bit Pro

[Krzysztof Grzelak]

Match KINGSROW - SCAN (2-move ballots)

Kingsrow 1.61 vs. Scan 3.1 1 wins, 0 losses, 157 draws, 0 unknowns

Kingsrow 1.61 x64

Opening Book = Best Moves
HashTable Size = 512 MB
DB cache Size = 30000 MB
The base ends = 8 Pieces
Threads = 8
Time = 1 Min / 75 Moves

Scan 3.1 x64

Variant = normal
Book = true
Book-ply = 4
Book-margin = 4
Threads = 8
Tt-size = 26
Bb-size = 6
Time = 1 Min / 75 Moves

Match played on a computer with the equipment.

Processor - AMD Ryzen Threadripper 1950X
Hard disc - Samsung 860 Pro SSD 1 TB
Memory of frames - 128 GB DDR4 2400
System - Windows 10 64 bit Pro