Vibe Coding

[BertTuyt]

These days programming has changed quite a bit due to Vibe Coding, which uses AI.
In my case I started with Codex from OpenAI, to develop a new GUI.
I have reported several times about (some) progress.
As this was (quite) successful, I started with a new challenge, tackling the evaluation function.
In some posts I will describe progress (if any).

First I received from Joost a dataset with around 100M labeled FEN's. The labels contain the outcome of the respective game and were limited to W, D and L (win, draw and loss).
Next I asked Codex to develop a training application with input the data-set, and a description of the neural network.
In my (first) case this was 190 inputs, and then 4 fully connected layers with 256, 32, 32 and 1 neuron(s).
To secure that the inputs only/mainly covered positional information, the material component was "hardwired" / added to the output of the last neuron. This can be changed in the future (for example a simple NN), but as a first guess/start the values man = 1.0, and king = 3.2 were used (not game-phase dependent).
Output was a file with the 32-bit float weights for the neural network.

See some screenshots of the tool developed by Codex, and named quattice

Bert

[BertTuyt]

The next step was to optimize NNUE.
Although NNUE is an Efficient Updateable NN, I wanted to get rid off the incremental update, and the need to copy the 256 * 4 byte float accumulator at every new node. For this reason Codex designed the quasar application which converts areas of the draughtboard (8 in total) to neural network tables (each with 256 floats). The base idea is to calculate every combination of white/black man in every area, and to add all the 256 byte floats for every piece/square combination to the table entry. In contrast with the patterns in Scan these areas are non-overlapping (and don't need to cover a logical draughts related concept). The index of the tables entries is calculated in a similar way as Scan.
The quasar application converts the .qnn file (22k KB) to a .qtf file (8913 KB).

See below screenshots.

Bert

[BertTuyt]

Last but not least when the .qtf file was available, Codex was given the assignment to take the code of the damage161 engine, and to convert this to a new damage184 engine, in which the previous evaluation was replaced by an evaluation designed by Codex, which could handle the .qtf file, understands the interfaces towards the engine, and uses the avx2 vector instructions. This was all information given to Codex.

In these 3 steps (the previous 2 provided in the previous posts), Codex was able (with some discussion between the 2 of us), to start with a labeled FEN file, and ended with a updated damage application.

As the proof of the pudding is in the eating, I started a DXP match against Kingsrow. Settings 5 minutes for 80 games, 6p Database, no book for Damage and a book for Kingsrow. Also only 1 core was used.

DXP match is in progress, but current status after 63 games, 3 wins for Kingsrow, and 60 draws.
One results was unknown, but i could see from the score that this was a loss for Damage
So not superior to Kingsrow, but to my opinion an encouraging result.

Bert

[gwiesenekker]

Hi Bert,

Interesting. Were the patterns suggested by Codex? In GWD I get the best results with the 8 '6x4' (12 squares) overlapping men-only patterns: 01 02 06 07 11 12 16 17 21 22 26 27..04 05 09 10 14 15 19 20 24 25 29 30..21 22 26 27 31 32 36 37 41 42 46 47..24 25 29 30 34 35 39 40 44 45 49 50. All possible occupations (dimension 3^12) are mapped on embedding vectors of length 16. The 8 positional-embedding vectors are added, passed through GeLU activation and are inputs for a dense layer of 16 neurons. For the material balance I use the four material-'patterns' of dimension 21 for the men and dimension 6 (max 5 kings) for the kings and a correlated material-'pattern' of dimension 21 * 6 * 21 * 6. All possible occupations are mapped on embedding vectors of length 64. The 5 material-embedding vectors are added, passed hrough GeLU activation and are inputs for a dense layer of 16 neurons. The outputs of both the positional and material dense layers are concatenated giving 32 inputs for a final dense network of 16x1 (or 8x8x1) neurons.

GW

[BertTuyt]

Gijsbert, yes the "patterns" were selected by Codex.
The match has ended, Kingsrow won (as expected).
Final result: Kingsrow 8 wins, 150 draws.

See the attached dxp file.

Bert

[Sidiki]

Hi Bert,

Great match, and nice to see that Damage upgraded to 18.4.
Congratulations.

Friendly, Sidiki

[BertTuyt]

Sidiki, I don't think that damage 18.4 is already better as the current damage 16.1 (which is used in all tournaments these days).
This exercise was mainly meant to see the potential of AI.
Impressive, but AI will not (yet) develop or improve engines towards or beyond the Kingsrow level.
But I'm sure the future is inevitable, this will happen (and maybe sooner as we think).
Also will AI support and help in generating the dataset which is crucial for learning.
And with all this AI power one can think what the future of man-made engines will be.
So I'm glad I was there from the beginning and hope I see the end.......

Bert

[Joost Buijs]

Also will AI support and help in generating the dataset which is crucial for learning.
And with all this AI power one can think what the future of man-made engines will be.

In my view vibe coded engines are not man-made anymore. You could also argue whether vibe coded engines should be allowed in tournaments or not. Vibe coding is more like a form of 'hyper cloning', LLM's are very good at combining existing code-bases and repositories they steal from everywhere, but they are not capable of creating anything new.

[BertTuyt]

Joost, you are right, Vibe coded engines are not man-made.
Also I support the idea that LLM are not (yet) capable of creating anything new.
But I'm not sure this will be a situation which we will see forever, or that at one point in time LLM's (or a successor with some new technologies) also get this creative capability.
The good news, we can see it all happening today.

Bert

[Joost Buijs]

Bert, I guess we just have to wait and see.

Currently companies are putting billions (even trillions) of dollars into AI-fabrics, when AI doesn't live up to expectations there is a big change that at some point they will fail. At least I won't put my money on it.

If they fail, the upside is that memory (and other hardware too) will get cheaper than it ever was before.

Joost

[Sidiki]

Hi all,

This's a very interseting topic.

I think you raise a very interesting point, especially regarding AI-assisted dataset generation. I agree that AI will play an increasingly important role in creating, filtering, and analyzing training data.

That said, I would make a small distinction between using AI as a tool and letting AI build an engine by itself. In my opinion, the vision, the architecture, the experiments, and the final decisions still come from the developer. The AI simply accelerates implementation and helps explore ideas more efficiently.

I also think that innovation often comes from combining existing ideas in new ways. Humans have always built on previous work, and engine development has never been an exception. AI certainly makes this process faster, but the creativity still lies in deciding what to build, what to keep, what to reject, and how to evaluate the results.

It will be interesting to see how the community views this in the future. Personally, I see AI as another powerful development tool rather than a replacement for the engine author.

By the way, I find it fascinating that more and more engines are embracing AI, not only in evaluation but throughout the development process. It seems the field is evolving quite rapidly.

Friendly, Sidiki

[BertTuyt]

Joost/Sidiki,

think the discussion is good, and we will certainly continue with that.
Where (I guess) all agree is that AI is great in quickly (?) writing a GUI and tools.
I wanted for example to better understand different networks made by AI.
Therefore I asked Codex to write me something, see picture.

Bert

[Joost Buijs]

Hi all,

This's a very interseting topic.

I think you raise a very interesting point, especially regarding AI-assisted dataset generation. I agree that AI will play an increasingly important role in creating, filtering, and analyzing training data.

That said, I would make a small distinction between using AI as a tool and letting AI build an engine by itself. In my opinion, the vision, the architecture, the experiments, and the final decisions still come from the developer. The AI simply accelerates implementation and helps explore ideas more efficiently.

I also think that innovation often comes from combining existing ideas in new ways. Humans have always built on previous work, and engine development has never been an exception. AI certainly makes this process faster, but the creativity still lies in deciding what to build, what to keep, what to reject, and how to evaluate the results.

It will be interesting to see how the community views this in the future. Personally, I see AI as another powerful development tool rather than a replacement for the engine author.

By the way, I find it fascinating that more and more engines are embracing AI, not only in evaluation but throughout the development process. It seems the field is evolving quite rapidly.

Friendly, Sidiki

Hi Sidiki,

It is true that several things such as Mini-Max, Alpha-Beta, and Transposition-Tables were developed at least 60 years ago and are still used in modern engines today.

The development of a good chess/draughts engine is a lot of work; this can take months to years to complete. With the help of Codex, Claude Code, and other AI models, it is possible to code a full engine in just a few hours. It will not play at top level, but probably better than engines from 10 years back.

Ask Codex to make a bitboard move-generator for draughts in C++, and with a bit of a question-and-answer game, it will be right in front of you within a few minutes. Ask Codex to make a neural-network evaluator for draughts, same thing. Ask Codex to make a multi-threaded search with a transposition table, same thing.

In my eyes, it is an undesirable situation that people who have never programmed before are able to put something working together within a few hours, something that other people have invested years of their time into.

Joost

[Sidiki]

Hi Joost,

I started, to expose my case, coding a fear years ago, 2006. But learned more via this forums and my many personal projects. To sa that i have a good knowledge in C++, just to quote it.

I completely understand your point, and I have a lot of respect for the people who spent years developing the foundations of modern engines. Without that work, none of today's AI tools would be nearly as useful.


However, I think there is an important distinction between building an engine and building a strong engine.

AI can certainly generate a bitboard move generator, a neural-network evaluator, or a transposition table. But asking for these components is only the beginning. Knowing how to combine them, tune them, generate the right training data, choose the right experiments, interpret the results, and improve the engine over time still requires a good understanding of the domain.

In my opinion, AI has lowered the entry barrier, but it hasn't eliminated the need for knowledge. Someone with no programming or engine-development experience may obtain a working engine quickly, but taking it to a competitive level is a completely different challenge.

I see AI as a powerful tool that accelerates development, much like modern compilers, debuggers, profilers, and version-control systems did in previous decades. The developer is still the one making the important decisions

Friendly, Sidiki

[BertTuyt]

Sidiki, I tend to agree with you.
I'm sure you can quickly build something, but it will not beat the top-programs.
And also in my case that real differentiator was the dataset provided by Joost, most likely not the code generated by AI.
And for now it is a good assistant, which needs however still some guidance.
So as you named it a new tool for developers.
Like the internet and forums helped programmers to very quickly share/discuss ideas and exchange sources.
This has certainly helped.
When I started with computer draughts the tournaments in the past were a great place to meet fellow programmers, and share the latest insights.
Nowadays things are quite different.
Within this context we are all building and standing on the shoulders of giants......

Bert