New NNUE Checkers Engine "DarkHorse"

[CheckersGuy]

Hello everyone,

Seeing that Cake had recently been released on GitHub motivated me to finally wrap up this project and upload it as well. I originally started working seriously on my checkers engine during the COVID pandemic, but sometime afterwards I gradually lost interest in programming, and the project ended up sitting unfinished. At that point the engine itself was essentially complete—I think only parts of the CheckerBoard GUI interface were still missing.

The author of Cake apologized for the state of his repository, saying he wasn't a proper software engineer and didn't "speak Git well." I can definitely relate to that sentiment. I'm not a software engineer either, so please don't judge the code too harshly—I only ask that you judge the engine instead.

The source code is available here:
https://github.com/CheckersGuy/DarkHorse

The precompiled DLL for CheckerBoard (place it in the CheckerBoard engines folder) can be downloaded from the release page https://github.com/CheckersGuy/DarkHors ... eleaseV1.0. At the bottom of the release page, you'll find the DLL. Only the DLL itself is required in the engine folder, since the network weights are already included inside the binary using incbin https://github.com/graphitemaster/incbin.
At the moment only an AVX2 build is available. It should run on virtually any processor released since 2013.

As far as I know, GUINN (https://github.com/jonkr2/GuiNN_Checkers) is currently the only other publicly available checkers engine using a neural network. Here are some of the engine's main features that are different compared to GUINN:

• The engine uses three neural networks:
  - a large evaluation network for position evaluation,
  - a policy network for move ordering,
  - and a much smaller network that is activated in winning positions to estimate the number of plies until a win or loss.
• Quantized inference to improve evaluation speed. The input layer uses 16-bit weights and 32-bit biases, while all remaining layers use 8-bit weights and 16-bit biases. As far as I know, GUINN does not currently use 8-bit weights in its quantized network.
• Fast inference by exploiting activation sparsity after the input layer. Most activations become zero after the first layer, allowing many computations to be skipped. An excellent description of this technique can be found in the Stockfish team's NNUE documentation:
  https://github.com/official-stockfish/n ... cs/nnue.md
• Bucketed weights. Beyond the input layer, the network's weights are split into buckets indexed by piece count, so the engine effectively uses a different set of learned weights depending on how many pieces remain on the board
• The evaluation network is considerably larger than the one used by GUINN, with about 1.3 million parameters. I mainly experimented to see how large I could make the network while retaining most of its playing strength. The architecture is probably not optimal, but training these networks takes a significant amount of time—and a fair bit of "alchemy"—so I never explored the design space systematically. I even trained a version with roughly three times as many parameters, and surprisingly it wasn't dramatically weaker even at bullet time controls (100 ms per move).

I also ran a short bullet match against KingsRow using 11-move ballots:

Match Results against KingsRow 1.19e (11-move ballots)

DarkHorse AVX2 v1.0 vs KingsRow 1.19e

224 wins, 61 losses, 3955 draws

GUI = CheckerBoard 1.77
Opening Book = None
Hashtable = 128 MB
Endgame Database = 2048 MB
Allscores = Off
Solve Mode = Off
Database = 8-piece WLD
Threads = 1
Time = 0.1 seconds per move

The match was played on an AMD Ryzen 7 9800X3D.

I hope someone finds it interesting, and perhaps useful as a reference for other neural-network-based checkers/draughts engines. If you decide to give it a try, I'd be happy to hear any feedback.

There are still a few features missing compared to KingsRow or Cake. For example, DarkHorse currently does not support an opening book, multithreading, or DTW/MTC endgame databases. Training the neural networks takes quite a bit of time, so if I ever get around to training a new one, I may release another version of the engine. If I do, I'll most likely include support for the missing features as well.

[BertTuyt]

Very interesting.
Can you share more details related to the training process.
How many games/positions, which positions did you exclude.
Did you use a CPU or GPU for training, how long did it take.
Also very interesting that you use a policy network, how did you train that one, and do you have any insight in accuracy?
I assume you only include non-captures and man-move only in the policy network , or do you include king-moves?

Bert

[BertTuyt]

I also see MCTSEngine, did you experiment with MCTS, policy networks, and NNUE?
In my case (and also other attempts) I never succeeded to make MCTS work with 10x10 international draughts, due to the problem of "traps". Do you have a similar experience?

Bert

[CheckersGuy]

Can you share more details related to the training process.
How many games/positions, which positions did you exclude.
Did you use a CPU or GPU for training, how long did it take.
Also very interesting that you use a policy network, how did you train that one, and do you have any insight in accuracy?
I assume you only include non-captures and man-move only in the policy network, or do you include king-moves?

Training process: Positions come from self-play games at a very fast time control — 1ms per move. I kept generating until I had about 800 million unique positions. Duplicates and any position with a capture available were excluded from the dataset.
For the self-play data generation, I use multi-PV search (what Kingsrow calls "All Scores") to increase move diversity. Instead of always playing the engine's top choice, I compute a probability distribution over the candidate moves and sample from it, so games don't collapse into the same handful of lines.
Training itself runs on GPU. I define an "epoch" as 100 million positions — I believe Stockfish uses the same convention and calls it a "superbatch." On my RTX 3060 Ti, one epoch takes roughly 7 minutes. The current eval-network was trained for 176 epochs on the value of the 1ms search, not the game result.

Policy network: The one currently bundled in the binary is an older version, and unfortunately I no longer have the training checkpoint for it, so I can't measure its accuracy easily without writing some extra code. A newer policy network trained on the current dataset gets around 54–55% top-move accuracy on the training-set. I did not have a validation set for the training-process.
The policy network is trained to predict the best move, and the dataset excludes captures — so king moves are included. Each non-capture move is encoded as an origin square (0–31) combined with a direction (0–3), giving 128 possible outputs. Some of those indices are never actually used, but I never bothered implementing the better encoding. The impact on speed is negligible.

I also see MCTSEngine, did you experiment with MCTS, policy networks, and NNUE?
In my case (and also other attempts) I never succeeded to make MCTS work with 10x10 international draughts, due to the problem of "traps." Do you have a similar experience?

Good catch — I must have missed that binary while cleaning up the GitHub repo. To answer your question: yes, I did experiment with MCTS at one point, but I couldn't get it to work well. It was a useful exercise for learning MCTS, but the resulting engine was far too weak to be useful for data collection. That said, it was a fairly short experiment, so I can't say with much confidence whether MCTS is fundamentally unworkable for this — just that my attempt didn't pan out.

[Ed Gilbert]

Robin, congrats on DarkHorse, it sounds like it is very strong, and I am looking forward to trying it. My computer has been tied up recently building more databases, but it will be finished soon and then I can start doing some of the other projects that I have been postponing.
Thanks for sharing your source code on GitHub. I should do that also -- another project. I will take a look this week and maybe email some comments.

-- Ed