Feike Boomstra Match: a unique man-machine event

[A.Presman]

On sunday November 15, 2015, a man-machine match took place between nine computer draughts programs (and a Frysian draughts program) and a team of strong human draughts players.
Unfortunately, the tenth program, TDking (winner of the 2013 Computer Olympiad), had to cancel.
The match was a side event to the draughts World Championship tournament, which is currently being played in Emmen, The Netherlands.
The man-machine match was organized in memory of Feike Boomstra, who died in 2010 and was the author of the strong draughts program BoomstraDam (later Horizon).
Remarkable is that Feike is also the father of Roel Boomstra, one of the candidates to become the next draughts world champion. Roel and his family were present at the start of the event.
This unique man-machine event was organized by Jaap Bus (referee), Leo Nagels (FCerberus) and Herm Jan Brascamp (draughts club Hijken DTC).

There were two rounds on one day.
The playing tempo was 75 moves in 2 hours for the human players and 75 moves in 1 hour for the computer programs. After 75 moves the position will be assessed, if necessary.

So we have two factors to consider if we want to compare human and computer playing strength. Firstly, the humans have twice as much time to think, and secondly, two games in a row means

the humans get more tired during the second game.

For the ratings of the human players I use the reliable rating list of the Dutch draughts federation, KNDB.
The KNDB rating system is an Elo-based rating system. But it has a different base than for instance chess ratings (which go as high as 2850), with the following rules of thumb:

Code: Select all

     >1500: international grandmaster (GMI)
 1400-1500: international master (MI)
 1300-1400: federation master (MF)
 1200-1300: major league player
 1100-1200: strong club player
<1100     : provincial level

The highest ranked human player is currently former world champion Alexander Shvartsman with a rating of 1574. World champion Alexander Georgiev is third (1556) after Roel Boomstra (1561).
(Virtual rating list of November 17, 2015: http://toernooibase.kndb.nl/)
In 2012, Shvartsman played a 6-game match against Maximus, which he won by 7-5 (+1 =5). He estimated the rating of Maximus to be about 1500, in accordance with its match performance rating (1501), see http://jhorssen.home.xs4all.nl/Maximus/ ... aximus.pdf.
Maximus has improved moderately since then, but is still less strong than the current top-3: Scan, Dragon and Kingsrow.
And then there are Damage and Damy, both not participating.

Of course, the best players of the world were not playing the machines - they were playing each other in the next room. But still, the human team consisted of very strong players, including

several masters and grandmasters.

Here are the results:

Code: Select all

  program      | round 1 opponent       title rating res. | round 2 opponent       title rating res. | perf.rating
---------------+------------------------------------------+------------------------------------------+------------
1 Sjende Blyn  | black v M. Stempher    MF    1325    2   | white v J. Okken       MI    1384    2   | 1755 *
2 Scan         | black v N. Hoving            1346    1   | white v H. Jansen      GMI   1462    2   | 1595 
3 JDraughts    | black v H. Jansen      GMI   1462    2   | white v J.M. Drent           1345    1   | 1595 
4 Kingsrow     | black v M. Kroesbergen MI    1421    1   | black v J.E. de Vries        1363    2   | 1583 
5 Maximus      | black v A. Domchev     GMI   1436    1   | white v N. Hoving            1346    2   | 1582 
6 Moby Dam     | black v W. Wolff       MF    1310    1   | white v S. Buurke            1245    2   | 1469 
7 Dragon       | black v A. Scholma     GMI   1470    1   | white v M. Kroesbergen MI    1421    1   | 1446 
8 Flits (iPad) | black v H. Kalk        MF    1243    1   | white v W. Wolff       MF    1310    0   | 1086 
9 Slagzet.com  | black v J.M. Drent           1345    0   | white v H. Kalk        MF    1243    0   |  894 *
                                                     --                                         --
                                             (1373)  10-8                               (1347)  12-6

*) for 100% / 0% scores, a performance rating difference of +/-400 is used (instead of >735 or even >1200).

Overall result: Machine-Man 22-14 (+7 -3 =8). Performance rating of the machines: 1438.


If we exclude the games of Flits (mobile hardware, no databases) and Slagzet.com (little effort to optimize playing strength) the result is 21-7 (+7 ; =7), a performance rating of 1572.


Statistically this is not much to go on, but it seems that -in contrast to chess- the very best players at draughts are still a match for the machine.
In-depth analysis of the games by human experts may shed more light on the matter.

Code: Select all

Frysian draughts:
1. Marten Walinga - FCerberus  2-0
2. FCerberus - Marten Walinga  0-2
Note: There exists at least one much stronger Frysian draughts program: Lusoris.

For more info see http://wcdraughts.com/?page_id=1465&lang=en
For the games see http://toernooibase.kndb.nl/opvraag/sta ... afko=23&r=

Jan-Jaap van Horssen (Maximus)

[Jelle Wiersma]

Good post Jan-Jaap!
1572 would be a 2nd place in the rating list.
When I do include Flits (a champion!) in your overall result, I get an average TPR of 1518
(Average opponent rating of 1368 + "Rule of 400" delta rating of 150).
1518 would be a 15th place in the rating list.

[jj]

Thanks, Jelle.
I did not include Flits because on an iPad Air 2, I estimate it to be 150-200 Elo points less strong than on a fast laptop/desktop computer.

[BertTuyt]

Thanks for posting.

I was able to view the match from Switzerland, and it was a pity I was not able to join.
At least we are approaching the world champion, step by step.

What was the Hardware used by the programs?
Think (as an example) Scan used a notebook, so there might be room for improvement.

These days the Damage hardware is based on a 8-core Intel processor ( i7 5960X) clocked at 4 GHz (and with water cooling).
Next year I think an Intel (consumer) 10-core processor (Broadwell-E) with be launched.
So we might close the gap soon.

Bert

[Krzysztof Grzelak]

Thanks for posting.

I was able to view the match from Switzerland, and it was a pity I was not able to join.
At least we are approaching the world champion, step by step.

What was the Hardware used by the programs?
Think (as an example) Scan used a notebook, so there might be room for improvement.

These days the Damage hardware is based on a 8-core Intel processor ( i7 5960X) clocked at 4 GHz (and with water cooling).
Next year I think an Intel (consumer) 10-core processor (Broadwell-E) with be launched.
So we might close the gap soon.

Bert

Bert need to ask an important question whether it is worthwhile to buy more and newer equipment. According to me - no, you have a good example after the Olympics.

[BertTuyt]

Thanks for your post.
I don't understand your reply.
I think that in general programs benefit from faster hardware.
So although there is certainly diminishing returns, it might be that there is still some advantage.
But I could be wrong.....

Bert

[Krzysztof Grzelak]

You were the Olympics Bert and took care of the other playing a very strong hardware. Compare your equipment with equipment Fabien. You have to ask an important question, is it worth it to buy equipment when you can win a tournament on the equipment much cheaper. That's what I mean.

Krzysztof.

[BertTuyt]

Now I understand.
For now my hardware is ok, so I need to improve the software.
There I agree with you.

But my thinking was, if programs like Scan (and others) also use faster hardware, they might also see the benefit.
And it might be that with the additional ELO gain they wil be close to world champion level.
In my case, I wont buy new hardware the next 3 - 5 years.

Bert

[Krzysztof Grzelak]

What to Scan, then I agree that better equipment will reach better results. What your software also agree that you need to improve software. But what happens when some great hardware, while the hardware lose a lot worse.

Krzysztof.