Vibe Coding

[Joost Buijs]

This is a nice story about vibe coding: https://brandonharris.io/Garbage-Pail-Code/

[Ed Gilbert]

This is a nice story about vibe coding: https://brandonharris.io/Garbage-Pail-Code/

Thanks Joost, that was very entertaining!

[MichelG]

Bert, I guess we just have to wait and see.

Currently companies are putting billions (even trillions) of dollars into AI-fabrics, when AI doesn't live up to expectations there is a big change that at some point they will fail. At least I won't put my money on it.

If they fail, the upside is that memory (and other hardware too) will get cheaper than it ever was before.

Joost

The companies might fail and go broke because they spend too much, but AI itself will not fail. For me, AI is a fact of life. It is here. They are smarter than me. If they never get smarter than they are now, that would be ok, it is smart enough.

I use AI up to 8 hours per day on my job. Projects that would have taken me months we can do in weeks or even faster. Our changelog is exploding with new features we produce.

In our software, there are bugs that for years we haven't been able to find or reproduce. We gave up on them, and accepted the occiasional crash. Now Fable reproduces and then solves such bugs in minutes.

Handcoding has been fun for the last decades. But it is a thing of the past, like analog photography. You still can do it, but it is just for the fun of it.

Michel

[Joost Buijs]

The companies might fail and go broke because they spend too much, but AI itself will not fail. For me, AI is a fact of life. It is here. They are smarter than me. If they never get smarter than they are now, that would be ok, it is smart enough.

I use AI up to 8 hours per day on my job. Projects that would have taken me months we can do in weeks or even faster. Our changelog is exploding with new features we produce.

In our software, there are bugs that for years we haven't been able to find or reproduce. We gave up on them, and accepted the occiasional crash. Now Fable reproduces and then solves such bugs in minutes.

Handcoding has been fun for the last decades. But it is a thing of the past, like analog photography. You still can do it, but it is just for the fun of it.

Maybe AI knows more, but that doesn't necessarily mean that it is smarter. I agree that bugs are sometimes difficult to find because a human always tend to overlook these. Fable seems to do a good job, there is nothing wrong with that, but I'm very reluctant to have AI generate pages of code for me.

I agree, for things that don't have to be highly optimized like a GUI, code generated by AI is good enough, but for things that have to be highly optimized like a chess engine, or a draughts engine, it is not good enough yet.

[MichelG]

AI can certainly generate a bitboard move generator, a neural-network evaluator, or a transposition table. But asking for these components is only the beginning. Knowing how to combine them, tune them, generate the right training data, choose the right experiments, interpret the results, and improve the engine over time still requires a good understanding of the domain.
Friendly, Sidiki

Still, AI is breaking into this as well.

I asked codex to write a new basic alpha/beta function and then list all the ideas that are used in chess engines. It came up with 12 ideas.

Then i asked to try them all, run the required experiments, keep the code if it found and improvement, and discard it otherwise.

It build and tested all ideas. First idea 1, play 50000 games against the original, and it went on with idea 2, etc.

It took all night, and in the end, the a/b function it created was on par with the one in dragon.

The point is, Codex came up with the ideas (well, as learned from humanity), did all the implementations, did all the experiments. All it needed was a single prompt that pointed it in the right direction.

As for optimisations, in my experience AI excels in it. In some cases i tell it to go and optimize a function. It will have ideas, test them by creating experiments, and then accept them or reject them. And it can keep trying until you are out of tokens.

For now we still need a mix between human input and AI as a tool. Humans outsmart the AI on general concepts and architecture, and are better in having novel ideas. AI can be really dumb sometimes. But the shift to AI-only will continue.

To me, it is exciting to see this revolution first hand, but it is also a bit scary. I wonder how the roles of human vs AI will continue to play out in the coming years/months.

[Sidiki]

Hi Michel,

I completely agree with your observations. What you describe is exactly where AI is becoming incredibly powerful: implementation, experimentation, optimization, and iteration at a scale no human can match.
My point was not that AI cannot build a strong engine. Clearly it can, especially with tools like Codex. My point is that someone still has to define the objective, provide the framework for evaluation, and understand what the results actually mean.
In my own project, the biggest performance gains did not come from asking AI to write an alpha-beta search. They came from decisions about training data, error learning, reinforcement mechanisms, testing methodology, and continuous tuning. AI accelerates all of this, but the direction still matters.
I also believe the balance will continue to shift toward AI. The question is no longer whether AI can code an engine, but how much of the research cycle it will eventually automate. We are probably witnessing the beginning of that transition.

Friendly, Sidiki

[BertTuyt]

Another iteration with Codex.
This time the float network (256x32x32x1) was converted into an int16.

158 games, 5 min/80 moves, 6p db, 1 core, both book.
Result 2 wins Kingsrow - 156 draws.

I don't think the program is really better, this is all within the statistical margins, but at least the depth has increased which always helps.

(Kingsrow did not include the last draw game).

Here the dxp file.

dxpmatch_20260724.pdn

(156.71 KiB) Downloaded 5 times

Bert

[gwiesenekker]

Hi Bert,

What are the results if you use 1 minute/75 moves? If I use 5 min/80 moves GWD draws all the time against Kingsrow, with 1 minute/75 moves sometimes GWD loses a game (but also sometimes wins).

GW

[BertTuyt]

Gijsbert,

nope so far i only tested with 5 minute games.
The background, with the reduction in depth, and the current network speed (and the processor I use), I often stay in the critical depth-area for Damage with 1 min games.
Therefore my focus is to increase nodes/second with the current network, or find a network with better quality.
Then at one point the 5 minutes games should end in 158 draws, which is then the signal to go to another time setting.
But as long as I lose games with 5 minutes, I have other tasks with higher priority.

Bert

[BertTuyt]

Michel, your new Dragon engine development (think version 5.0), is that a cooperation with a tool like Codex (or Claude)?
And if so, can you share your experience, and in what way is the tool helping you?

Bert

[MichelG]

Michel, your new Dragon engine development (think version 5.0), is that a cooperation with a tool like Codex (or Claude)?
And if so, can you share your experience, and in what way is the tool helping you?

Bert

Good question, but not easy to answer.

Coding has become hard to define. Codex does things, i do things. I check, i experiment, codex experiments, codex checks. It is one big evolving mess

The main thing you have to learn, working with AI, is when you can trust it, and when not. If you can't trust it with a prompt, subdivide it into smaller chunks.

For my work, codex wrote a 20000+ line program, and it is complex beyond my understanding, flawless and beautiful. In other instances, it is unable to align 2 labels properly. You need to be aware of where you bump into the limits.

I knew the 20000+ line program would work in a single prompt, because it could verify the result.

So i take the biggest chunk of work that: 1. Codex can create reliably, and 2. I or codex can verify in some way. Sometimes it is big, sometimes small.

For dragon, codex has helped me rewrite the old algorithm for learning the weights from CG to LBFGS and learn weights 10x faster. It has been finding optimisations that i had not thought about. And has experimented with and finetuned the learner.

It found and fixed dozens of bugs, it removed thousands of warnings.

AI writes most of the lines, but i still feel it is me coding.

here are some prompts i recently used.

Investigating Krzysztof's problem:

In const.h, there is this line:
#define MAXPROCESSOR 40 // note that iterator will fail if max>45
See if it really will fail at >45. Explain why, and do a suggestion on how to solve that

Making PGO easier to work with:

Build a skill for creating PGO executables.
It should create the instrumentation files. And then run this in the engine:
evaltype 0
run "D:\dragon-sampler\tests\pgo.txt" 1
evaltype 1
run "D:\dragon-sampler\tests\pgo.txt" 1
quit
Then build the PGO executable.
Do this for both the SSE2 and AVX2 executables.
After that, report the filenames of the executables that have been created

Gaining 1.5% speed:

The current popcount function: __forceinline popcount(BITFIELD x)
has a check on very old systems.
Instead add a check for the popcount ability of the cpu when the engine starts.
If it does not have it, message the user that its cpu is too old.
Replace the curent popcount function by a macro that calls the instrinsic

[BertTuyt]

Michel, interesting!

In my case I only experimented with neural-network learning and neural network evolution functions with avx2.
But I already smell that the way I code or program (and i agree the definition has changed), is very different from the past.
I'm still the architect and generate the main ideas, but the implementation, coding and (sometimes) testing is done by Codex.
I even inspect code, and sometimes I suggest Codex to change a little, as I don't want to manually modify Codex code.

Last week I asked Codex to examine and compare the search-function from Damage, Darkhorse and Stockfish, and to issue a report with possible ideas which are worth trying.
Still on my to-do list but interesting.

When OpenAI recently switched from 5.5 to 5.6 I even had the idea that Codex was really better as before, not sure you had the same experience.
And I'm convinced we will have the same feeling with future new versions, as this will not stop (maybe only accelerate).
The genie is out of the bottle, and will not go back....

Bert

[MichelG]

For me, Codex 5.5 was a big enabler. It allowed big functions or small programs to be written. After creating, maybe requiring a 100 prompts to tune to perfection.

5.6 was to first to enable whole programs to be written, without even needing to look at the source, and then tens of prompts to tune.

If i look at my job, things are a bit more complicated then my workflow for Dragon. Claude may take for instance 30 minutes to implement something, and you can't go stare at your screen for that time.

So, at times, i have up to 4 or 5 AI's working simulateously, on different tasks, provided they can run with limited verification. And sometimes only 1, if it complex and needs your constant attention. I use Claude to generate code, and sometimes Codex to do independent verification.

It takes a while to get used to working this way, and it also requires preparation in the form of creating skills and building connections for the AI, but i can see my production skyrocket.

[BertTuyt]

Michel, I'm really curious what Dragon 5.0 will bring to the table.....

Bert