Protocols and tools

[Fabien Letouzey]

Hi all,

I am preparing the Scan release and, following Moby Dam, have implemented part of DXP. I would like to know the current state of draughts protocols and external tools.

But first, a few words about the chess protocols. We all know CECP (xboard) and UCI. What not everybody knows is why there are two such firmly entrenched camps, so here's how I see it.

CECP is an event-based protocol. The engine is a black box (agent): events come in, events (mostly moves) come out. Programmers from this camp want to know as much as possible about the outside world and use words such as "real chess" (don't ask) or here: "engine in control". I guess the model here is to adhere as closely as possible to the rules for humans, going as far as calling the referee before claiming a draw.

UCI comes from the realisation that, from the engine point of view, a game is just a sequence of searches. We could call it a position-based protocol but I view it more like a search API, albeit in text form. Programmers from this camp (and I'm one of them) want to know as little as possible from the outside world in order to simplify the engine code. The game logic then goes out of the engine, presumably to a GUI or other tool. This allows more common code between engines, making access easier to beginners and veterans alike. In exchange, information is more limited.

In draughts, I see DXP and Guide; maybe there are others (or attempts) that I don't know of.

I expected DXP to be a very intricate, mostly binary communication protocol. While it does not look as modern as text-based protocols, I was surprised to see that it was in fact very carefully designed. Unfortunately, while it contains just enough to play games, it is lacking (among others I guess) engine parameters and search information. So I assume that the user experience becomes fairly limited, in particular for analysis.

Then there is Guide. I can see how it's loosely based on chess protocols with a few simplifications like reducing state. While it superficially looks like UCI, it also has a CECP feel to it.

I note that both protocols rely exclusively(?) on sockets for communication. This is painful to implement (not part of C/C++ standard libraries), especially in a portable way though Harm managed it. I don't understand why this is not left for external tools to do ...

---

All of this was only an introduction. What I would like to know is what came out of past discussions on the subject of protocols and external tools.

I can see that DXP is implemented in many engines and used a lot to play matches. What about the most important limitations (I noted engine configuration and search information)? I know about the time limit, but I'm looking for deeper, not easily fixable, concerns.

The state of Guide appears less clear to me. The document I have is 11 years old and I remember reading that the protocol was not "fully implemented". Furthermore it doesn't look like any other engine than Dragon and Damage have implemented it. Why not?

The tool side is also important. Ideally we should first focus on what people want to have/do (I read about tournament managers) and see how the implementation can be distributed among external tools (not necessarily graphical) and engines via the use of protocols.

Thanks,

Fabien.

[Rein Halbersma]

Hi Fabien,

The computer draughts community is much, much smaller than the computer chess community, with a lot less interaction (this forum e.g. is very quiet, and many regular participants to the draughts tournaments have never posted here). The lack of scale is why a lot of chess-inspired infrastructure is missing. Many people have written their own private GUIs (sometimes completely integrated with their engines, so not easy pluggable) for their own programs, and it's only through DXP that people have run engine matches.

I agree that the fundamental problem right now is lack of a uniform protocol. A protocol can act like a platform that allows GUI writers to meet engine authors. Right now, we have the PDN standard as a pretty mature protocol for exchanging draughts game notations. I think it's of the same (if not higher) quality as PGN. Unfortunately, for engine/GUI communication, we are very much behind.

Right now there are three candidate protocols to use as a starting point:
1) DXP: as you point out, it's carefully designed and successfully achieves its goals of platform independent engine matches. It's also very limited and would require a lot of new commands to be usable for general engine/GUI communication. The Dam 2.2. GUI is nice, but works but only for Windows (and not open source, so not easily portable to other platforms) and is also socket-based.

2) GUIDE: this is a nice text-based protocol and works for 2 engines already. The drawback is that there are only 2 Windows-based GUIs, which both are not stdio-based but socket-based. For me, GUIDE is not a viable option unless there is both a Linux binary (open source would be even better!) and the GUI become stdio-based instead of socket-based. See this post for how remote engines can connect.

3) CECP (i.e. WinBoard/XBoard). HGM has developed a WinBoard Alien edition that can play 10x10 draughts, but the user interface is a bit clunky for multiple captures (you need to press the Ctr button and click on intermediate squares twice). I asked him on Talkchess about smoother and more general support for draughts-like games (I am interested in other variations than just 10x10) and it seems that it requires more advanced multi-leg piece support and more advanced engine feature detection by the GUI (engines can communicate their variant). I don't think there are fundamental obstacles apart from gently convincing HGM to prioritize this because we as a community would greatly benefit (or we could write such support ourselves and submit it to HGM as patches).

I would prefer to use CECP because it works on all platforms (you can compile from source if necessary), is text-based through stdio instead of sockets, and opens up the entire tournament manager interface of the WinBoard/XBoard GUI as well.

Furthermore, the CECP would also allow draughts programs to use cutechess-cli to run engine matches more or less out of the box (because cutechess-cli has an xboard mode). In principle, it is also possible to run engine matches using cutechess-cli with GUIDE e.g. but then you need to hack those sources. It should be doable by deriving from their general ChessPlayer class and implementing the GUIDE protocol, but that requires quite a bit of work I think.

Rein

[Rein Halbersma]

I note that both protocols rely exclusively(?) on sockets for communication. This is painful to implement (not part of C/C++ standard libraries), especially in a portable way though Harm managed it. I don't understand why this is not left for external tools to do ...

Note that Boost.Asio provides platform independent socket support. It's going to be part of C++17 and is also available as a standalone library, see e.g. this chat message protocol example. It's easily generalizable to full DXP support.

But I agree, one should not have to do this

[Rein Halbersma]

The tool side is also important. Ideally we should first focus on what people want to have/do (I read about tournament managers) and see how the implementation can be distributed among external tools (not necessarily graphical) and engines via the use of protocols.

So more specificially, my top 3 wishes:

1) platform independent GUI (e.g. also on Android using a 10x10 version of Aart Bik's checkers GUI)
2) engine matches/tournament support (WinBoard/XBoard or cutechess-cli)
3) remote engine support (ssh + stdio based GUI, not writing socket-code myself)

Oh, and all that for arbitrary draughts variations and board sizes

[BertTuyt]

I tend to agree with Rein, that in general the computer Draughts community is far smaller and less active (at least on this forum), compared with the computer Chess world. Many programmers are already active between 20 - 30 years, and i can imagine that it is a huge time investment to re-implement something completely new. In this case Moby Dam and Scan was really an exception, and I think it is very positive that Harm and Fabien shared their code.

In my ( = Damage) case, the tools I use is the Visual Studio Community 2013 version (Microsoft), which is nowadays completely free (and quite soon the 2015 version will be released). Both the Engine and GUI use Microsoft Foundation Classes (MFC). If I would restart GUI development I would go for Qt, and maybe I will do this one day, but also for me, I have other priorities. Another workhorse is Notepad++, which I think is an essential tool for every programmer. Smaller tools are WinMerge and FreeFileSync.

As most programmers integrated GUI and Engine, and dont have or want to spent the time for a different design, only 2 programs so far use GUIDE, Damage and Dragon. As GUIDE was never really standardised, I guess that both Michel and myself, have made proprietary extentions. For me it was and is still logical to split these 2 components (Engine and GUI).

Nevertheless I would welcome a new standard, although I dont have a clear preference yet for the 2 solutions Fabien mentioned. Basically GUIDE is loosely based on both (as Fabien also recognized) and there is no clear strict underlying model and/or architecture.

But if we would go for standardisation I will certainly implement this standard in my Engine and GUI.
Basically for all interested the GUI source is open for everyone. I also agree to move socket handling towards the GUI, and implement simple input output functions in the engine itself.

Although I really like DXP (thanks to Frank, very logical structured architecture), as it has done a great job for enabling engine-engine matches, and most programms have implemented this, I in general dislike a binary protocol. So im not in favor in extending DXP, Im more interested in a text based protocol.

Last but not least, a tournament manager is also high on my wishlist.
And maybe i should consider a Draughts fork of an interesting Chess GUI with this function embedded.

Bert

[Wieger Wesselink]

I am a user of draughts programs for my own draughts analysis tools. I've implemented the DXP protocol in java, which was not too hard. But I don't really like it, since in general socket communication is quite cumbersome, and the protocol is way too limited for my purposes.

The CECP protocol looks quite promising to me. I like the simple text based approach, and it seems like the set of commands is adequate. So it would be great if draughts programs adopted something like this.

I'm looking forward to the Scan release! Especially if it can be run under Linux, it would be a very interesting choice for me.

[Harm Jetten]

Wieger, Scan has been released (see the Scan topic).
It can indeed be compiled and run under Linux as well.

[Wieger Wesselink]

Wieger, Scan has been released (see the Scan topic).
It can indeed be compiled and run under Linux as well.

Thanks a lot, I had missed that! Indeed it runs out of the box on Linux. This is really great, since it looks like now I can finally analyze games without having to switch to a Windows machine.

[Rein Halbersma]

Wieger, Scan has been released (see the Scan topic).
It can indeed be compiled and run under Linux as well.

Thanks a lot, I had missed that! Indeed it runs out of the box on Linux. This is really great, since it looks like now I can finally analyze games without having to switch to a Windows machine.

Both Scan and Moby Dam run on Windows and Linux.

[BertTuyt]

Another tool, available for Download and free Microsoft Visual Studio 2015 Community Edition.

Bert

[Rein Halbersma]

Another tool, available for Download and free Microsoft Visual Studio 2015 Community Edition.

Bert

Uhm: yes, because free compilers are nice. But also: no, because somewhat off-topic?

Let's abstain (or open another topic!) from discussing general programming tools / IDEs, and use this thread to focus on draughts specific infrastructure to help lift existing protocols and tools for engine/GUI communication and other analysis purposes to a higher level.

[MichelG]

I think Bert is right in that the draughts community is very small, and (a long time ago) i was happily surprised to me that even something like DXP even exists.

At the lowest level dragon uses memory mapped files and events to communicate between the gui and the engine. It still should support sockets as well, but i haven't tested that in years. At a higher level, dragon uses guide commands, but with a number of extensions.

In any case, both DXP and guide are insufficient to write a fully featured draughts-program. One example is that dragon has a option to show the evaluation of all moves for a given position. It is currently controlled by the engine, which sends updates to the guide. You implement this feature using guide by moving a lot of the code to the gui. (the gui would play move 1, get an analyses, then send move 2 and so on) But it would take a lot of effort on my part, and be less efficient than the current implementation.

Another issue is the perspective of the user. Where is chess, one engine can outperform another by hunderds of elo points, the top draughts programs can not. At 1 hour/game, the programs are probably only 100-200 elo points away from perfect play anyway. My guess is that for most users the strength of the program matters less than the features it offers, as long the strength is sufficient. Which makes a communication standard mostly a feature for programmers.

For me, dragon is just a fun hobby project which has 2 goals; play strong and offer a rich gui. With limited time resources i want to spend as much time as possible on those goals, and as little as possible to implementing complex protocols. Nevertheless, if at some time a new protocol is developed, i might support that.

Michel

[Maurits Meijer]

Can someone give me a link to the GUIDE protocol?
Information about DXP can be found at http://www.mesander.nl/damexchange/dxpmain.htm

Are pipes the preferred method for communication with external tools?

[Rein Halbersma]

Can someone give me a link to the GUIDE protocol?
Information about DXP can be found at http://www.mesander.nl/damexchange/dxpmain.htm

Are pipes the preferred method for communication with external tools?

Look here, in this old thread

[BertTuyt]

As I mentioned in this Forum , the Damage GUI has next to sockets the option to connect to an Engine with Pipes (since this week).
The advantage is that one does not need to add sockets to the engine , and that's simple stdin and stdout can be used.
So far I now can start Scan (with a hub command), and receive the Scan name and author.
I also include a debug window, so I can type specific Scan commands.

The first thing I want to work on is to process the move message (after the go).
Scan replies with a standard notion move ( such as 31-26) followed by (at least I assumed, didn't go into details) the first expected counter move.
In GUIDE the move format is from to { Captures [capture-1] [capture[2] ... }
If there is no capture the output is (as an example) 31 26 00.

As the engine exactly knows what move including all the captured pieces, this format also also solves the ambiguous move problem.
It is also true that in 99.99% most likely only from and to is sufficient.
One can also solve the ambiguous problem by an capture notation (including turns or captured pieces) as also is used in pdn.

My question , is there a specific preference we can standardize, so I can include this in the Damage GUI?

And is it evident that the Damage GUI will be made available to all interested.

Bert