NNUE

Discussion about development of draughts in the time of computer and Internet.
Post Reply
Krzysztof Grzelak
Posts: 1307
Joined: Thu Jun 20, 2013 17:16
Real name: Krzysztof Grzelak

Re: NNUE

Post by Krzysztof Grzelak » Fri Mar 25, 2022 22:41

BertTuyt wrote:
Mon Mar 21, 2022 13:06
Sidiki,

even KR is not perfect, although very close.
The win against KR was based upon a missed combination (by KR) towards a king.
Think this is related to the "heavy" pruning in KR, it is very balanced and most likely very optimized, but once in 100+ games a mistake can occur.
But anyway KR is the absolute benchmark, and although it might be possible to get on par, I assume that due to the strength, winning a 158 games match against KR with a large margin with normal time-control (and multi-core) is totally impossible.

In chess you see constant improvement, im afraid that in Computer Draughts we are already close to the maximum.
Nevertheless I hope that people are still motivated to improve their programs (although it is quite silent in this forum, compared with many years ago).

But the good news, i know that at least 2 people (under the radar screen) are working to implement a neural network in their program.

Bert
Who do you mean Bert.

BertTuyt
Posts: 1573
Joined: Wed Sep 01, 2004 19:42

Re: NNUE

Post by BertTuyt » Sat Mar 26, 2022 09:54

Krzysztof,

to which sentence are you referring?

Bert

Joost Buijs
Posts: 460
Joined: Wed May 04, 2016 11:45
Real name: Joost Buijs

Re: NNUE

Post by Joost Buijs » Sat Mar 26, 2022 13:10

Sidiki wrote:
Fri Mar 25, 2022 19:03
Yes, i think that, as in chess world that already got a very respectable level with learning. If the programers on this forum, very smarts put together they ideas, certainly, things will grow up quickly.
Joost, Fabien, Ed, Michel, Jaap and the others, please can you try the implementation of this programming option ?
Hi Sidiki,

Several weeks ago I started with a reinforcement implementation for International Draughts. It starts with a random network, plays a batch of games against itself, updates the network depending upon the outcome of these games, and repeats this over and over again. The problem with this is that you need a large amount of games to train the network before you get something decent out of it. The games also have to be played at a reasonable depth if you don't want to miss out on tactics, currently I use games of 1 second per move. The last two weeks I've played 180.000 games on my 32 core machine, this is still not enough to get very good performance.

The network already plays a decent game, it's better than the top programs from 5 years ago, I expect that it needs at least 10 times as much games before it reaches a level so that it won't lose anymore. In the mean time I found an improvement in the NNUE network architecture that I use that looks promising, I first want to explore this before I start playing another bunch of games.

For Chess you'll need something like 40 million games to train a network this way. I have the feeling that Draughts has a smaller parameter space, so probably 4 million games will suffice, but this is still an enormous amount of calculations.

Joost

Sidiki
Posts: 313
Joined: Thu Jan 15, 2015 16:28
Real name: Coulibaly Sidiki

Re: NNUE

Post by Sidiki » Sat Mar 26, 2022 22:31

Joost Buijs wrote:
Sat Mar 26, 2022 13:10
Sidiki wrote:
Fri Mar 25, 2022 19:03
Yes, i think that, as in chess world that already got a very respectable level with learning. If the programers on this forum, very smarts put together they ideas, certainly, things will grow up quickly.
Joost, Fabien, Ed, Michel, Jaap and the others, please can you try the implementation of this programming option ?
Hi Sidiki,

Several weeks ago I started with a reinforcement implementation for International Draughts. It starts with a random network, plays a batch of games against itself, updates the network depending upon the outcome of these games, and repeats this over and over again. The problem with this is that you need a large amount of games to train the network before you get something decent out of it. The games also have to be played at a reasonable depth if you don't want to miss out on tactics, currently I use games of 1 second per move. The last two weeks I've played 180.000 games on my 32 core machine, this is still not enough to get very good performance.

The network already plays a decent game, it's better than the top programs from 5 years ago, I expect that it needs at least 10 times as much games before it reaches a level so that it won't lose anymore. In the mean time I found an improvement in the NNUE network architecture that I use that looks promising, I first want to explore this before I start playing another bunch of games.

For Chess you'll need something like 40 million games to train a network this way. I have the feeling that Draughts has a smaller parameter space, so probably 4 million games will suffice, but this is still an enormous amount of calculations.

Joost
Hi Joost,
I read carefully, and you pointed something very important, you found an improvement in NNUE that make that Ares got a little more power.

Very interesting. And if 4 millions of games, i suppose, supervised are enough to reach a respectable level in international draughts engine, this's also a good new.

Bert and Joost, thanks.

If possible, the others members of the family are welcome 😎.

Thanks again to all the programers and God bless you all.

Friendly, Sidiki.

Krzysztof Grzelak
Posts: 1307
Joined: Thu Jun 20, 2013 17:16
Real name: Krzysztof Grzelak

Re: NNUE

Post by Krzysztof Grzelak » Sun Mar 27, 2022 07:08

BertTuyt wrote:
Mon Mar 21, 2022 13:06
But the good news, i know that at least 2 people (under the radar screen) are working to implement a neural network in their program.

Bert
I think I know at least 4 people.

Sidiki
Posts: 313
Joined: Thu Jan 15, 2015 16:28
Real name: Coulibaly Sidiki

Re: NNUE

Post by Sidiki » Fri Apr 01, 2022 22:55

Hi Joost, Bert, Ed, Jan-Jaap and the others,

In NNUE, is't possible in the supervising step, to, or before supervising step, permit to the program to learn a more agressive style of game, or another one depending of the programers ?

Because, Kingsrow, Damage 16.1 NNUE , and Ares are more classical, and Scan, Damage 16.2 and Dragon are more agressive.

Sometime, the problem with agressive style, it's that if the tactical step or seeing combinaisons aren't perfectly coded, this style fail against a classical style.

All the games lost by Scan, for just write on him, was by combinaison.

Friendly, Sidiki

BertTuyt
Posts: 1573
Joined: Wed Sep 01, 2004 19:42

Re: NNUE

Post by BertTuyt » Sat Apr 02, 2022 12:24

Sidiki, where the style comes from is also not perfectly clear to me.
Maybe the same with children with different characters, but raised in the same family :)

For supervised learning i assume that the program style is too a large extend determined by the training games.
Next to that the parameters in the search (like pruning, late move reductions) are relevant.

For reinforcement learning, where the network has initially random values, I guess its mainly related to search settings.

But not 100% sure, programs sometimes behave in a non predictive way, and thats also part of the fun.

Bert

Sidiki
Posts: 313
Joined: Thu Jan 15, 2015 16:28
Real name: Coulibaly Sidiki

Re: NNUE

Post by Sidiki » Sat Apr 02, 2022 14:28

Hi Bert,
:D, i see the idea of son of same family.
Effectively, the goal it's to play a decent game and win against weakers engines.

Also, i understand that command a playing line or style to an engine due to the largest moves possibilities as you said, it's another thing.

Again, it's just for fun.

God bless you and all the others, we are waiting for the differents update. 8)

Friendly, Sidiki

Joost Buijs
Posts: 460
Joined: Wed May 04, 2016 11:45
Real name: Joost Buijs

Re: NNUE

Post by Joost Buijs » Sat Apr 02, 2022 16:53

Hi Sidiki,

With supervised learning it depends upon the style of the program that was used to generate the training data. For Draughts there is no publicly available training data, so most people generate training data with the same program that they use for playing games. When the search of this program is good at tactics it will most likely develop a more agressive style. Of course it is possible to use data generated by other programs with a different style, unfortunately for Draughts there is almost nothing available. For Chess you can download millions (or even billions) of high level games played by the strongest programs, but that is another world.

The problem with NNUE still is that it is slower than patterns, with a good implementation maybe twice as slow, this hurts. In the near future there will be processors that are better at running tasks like these, this will mitigate the difference in speed.

Joost

Sidiki
Posts: 313
Joined: Thu Jan 15, 2015 16:28
Real name: Coulibaly Sidiki

Re: NNUE

Post by Sidiki » Sat Apr 02, 2022 22:39

Joost Buijs wrote:
Sat Apr 02, 2022 16:53
Hi Sidiki,

With supervised learning it depends upon the style of the program that was used to generate the training data. For Draughts there is no publicly available training data, so most people generate training data with the same program that they use for playing games. When the search of this program is good at tactics it will most likely develop a more agressive style. Of course it is possible to use data generated by other programs with a different style, unfortunately for Draughts there is almost nothing available. For Chess you can download millions (or even billions) of high level games played by the strongest programs, but that is another world.

The problem with NNUE still is that it is slower than patterns, with a good implementation maybe twice as slow, this hurts. In the near future there will be processors that are better at running tasks like these, this will mitigate the difference in speed.

Joost
Hi Joost,

Thanks again for taking time for us, as Bert and we hope the others in a near future.

If i understood, when the tactical step it's very good, the program will be agressive due to the fact that anything will escape to him.
I see the idea.

And as you said, theresn't enough games of others engines played at a high level.

I will expose 2 things or 2 questions, there are for everybody, you, Bert, Ed and the others.

1. The training step can be done with a second engine, ie a kind of dxp games.
Or is't possible to use games from dxp against others programs and then to supervise ?

2. The speed has the upper on the eval function, ie, if the eval it's good and the program can't see more nodes than another that haven't a better eval than him, this can create a serious problem.
Or in one sentence, what's is the link between the eval and the speed in NNUE or generally.

Friendly, Sidiki.

Joost Buijs
Posts: 460
Joined: Wed May 04, 2016 11:45
Real name: Joost Buijs

Re: NNUE

Post by Joost Buijs » Sun Apr 03, 2022 09:34

Hi Sidiki,

Yes, with reinforcement learning is is possible to have the engine play via DXP against other engines and to let it learn from these games.
With supervised learning you can use a database of games played by other engines, but you need games with enough decisive outcomes, only draws won't work. You also need games with a reasonable depth (otherwise you will miss tactics), this makes it very time consuming. Let's say 2 million games, 1 second per move, because the opponent has to move too this will take about 96 months, when you run 32 games in parallel (like I do) it still takes 3 months to do all the calculations.

About the speed I can't say anything, when the speed gets lower the engine usually gets weaker (if there is no other compensation like a better eval). The best Draughts engines are near the level that games will always be drawn, this makes it very difficult to predict anything about the influence of speed.

Joost

Sidiki
Posts: 313
Joined: Thu Jan 15, 2015 16:28
Real name: Coulibaly Sidiki

Re: NNUE

Post by Sidiki » Sun Apr 03, 2022 21:29

Hi Joost,

Thanks for this clear answer, i must admit that NNUE itsn't a very easy thing. I read about NNUE supervising and non or auto supervising. I don't know if you and/or Bert have already read it. https://medium.com/@callon84/an-introdu ... a340a6b227

For the speed, i know now and it's clear that a slow program will be without doubt weak.

God bless you and all the others.

Friendly, Sidiki

Joost Buijs
Posts: 460
Joined: Wed May 04, 2016 11:45
Real name: Joost Buijs

Re: NNUE

Post by Joost Buijs » Mon Sep 05, 2022 09:28

It seems that you don't need a very large NN to get good performance with International Draughts, the current NN of Ares has ~53K 16 bit weights, even a smaller NN with ~30K weights gives me good results. With NN evaluation and AVX2 SIMD code the search reaches ~40% of the speed I get with an optimally implemented 12 bit pattern evaluator, with AVX-512 SIMD code this figure is ~50%.

Of course it is possible to add tricks like lazy evaluation to increase the speed even further, however this is not very elegant so I decided to refrain from tricks like this.

Due to the lower speed (with games of 90 moves in 1 minute on a single core) there is a loss of about 13 Elo points, with faster games the loss is higher, with slower games the loss is less, this is due to the high drawing tendency of International Draughts. In the future there will be new CPU's with better instructions, this will enable us to do the matrix multiplications needed for calculating a network somewhat faster.

Another interesting development are binary NN's, they could be even faster than patterns, certainly something I'm going to look at.

Rein Halbersma
Posts: 1720
Joined: Wed Apr 14, 2004 16:04
Contact:

Re: NNUE

Post by Rein Halbersma » Mon Sep 05, 2022 15:07

Joost Buijs wrote:
Mon Sep 05, 2022 09:28
Due to the lower speed (with games of 90 moves in 1 minute on a single core) there is a loss of about 13 Elo points
13 Elo compared to what? What is the benchmark here?

Joost Buijs
Posts: 460
Joined: Wed May 04, 2016 11:45
Real name: Joost Buijs

Re: NNUE

Post by Joost Buijs » Mon Sep 05, 2022 17:58

Rein,

It is -13 Elo compared to the same program with a 2.5x longer thinking time. I ran several matches, each time the match came out between -15 and -11 Elo. Of course this doesn't mean that for another program it will be the same.

[edit]
I only wanted to know what the difference in playing strength would be if I could run the NN evaluator with the same N/s speed as the pattern evaluator. On my Intel machine the speed on the start position with pattern evaluation is ~13,3 Mn/s, with AVX2 NN evaluation it is ~5,3 Mn/s, with AVX-512 NN evaluation it is ~6.6 Mn/s. AVX-512 does not give as much gain as expected, the processor clocks down by a large margin when I run AVX-512 code on it.

Post Reply