Some time ago Jaap Bus asked me if it was possible to standardize on EndGame databases.
Jaap uses multiple computer programs , to analyze (for example) endgame compositions.
To name a few (and most likely this list is not complete):
- Truus, Damage, KingsRow, TurboDambase, Dam 2.x, Flits.
All these programs using different EndGame Databases which can not be exchanged between Draughts programs.
In the past this was maybe not an issue, as the (standard) DB was 6p and "only" occupied a few GigaByte (I only know that Gerard has a complete different approach which consumes far less memory).
However with 7P Databases , and 8P Databases (so far only Ed Gilbert, Kingsrow has calculated them, but i guess some other programs will also start preparing them in the near future), this will become an issue.
So my basic question to the programmers community, do we want to create a standard for EndGame Databases (even if we have a standard, a specific program could always use an "own" implementation next to the "default").
Maybe I forget a few details , but the "normal" implementation (and again so far I guess that only Gerard has a quite novel approach) is based on :
- Databases use initially 2 bits for a result (Win/Draw/Lose/Unknown), so we have to agree which bit-combination is used for which result.
- Naming convention line DB7-XXXOvOOX-77 (or whatever).
- Include conversion databases?
- Compression Technique used.
- Most DB's do not include the white-to move and capture positions (to improve compression).
- Index Function : Position --> DB-Index.
I think I have (most likely) overseen some additional boundary conditions, but im interested to see some reactions, and if there is a willingness to develop such a common standard.
With kind regards,
Bert
Open Standard for EndGame Databases
Re: Open Standard for EndGame Databases
Hi Bert,
Many comments can be made to your post :
1) As you mentionned yourself the approach I used in Damy is different as most of the other programmers. Let's take a very simple example. In the 5WK-2BK db almost all positions are winning positions for white. As a consequence the corresponding db in Damy is made of two files : the files of draw positions and the file of lost positions for white (this last file is empty if is white to move!).
As you can see I do not use a db based on 2 bits per position.
2) In addition why 4 states (Win/Draw/Lose/Unknown) instead of 3(Win/Draw/Lose) ?
3) For now several years, all world championship (and it will also be the case for the next wch in Brazil) use for tie break criteria the notion of draw+ and draw-.
I think that the rule for computers should be quite identical to the rule used in the major human tournaments. In that case we have to handle 5 states (Win,Draw+,Draw,Draw-,Lose).
I generated in the past the corresponding 5pieces db and I intend now to generate the 6p and 7p db.
4) Is compression so important ? For storing the 7p db with 5 states per position, without compression, we need about 450Gb. It was too high some years ago but it is not so high now. On my new computer I have a disk of 1,5Tb. For the 8p pieces db we need 8,5Tb. It is a little to high today but certainly not in 2 years!
Many comments can be made to your post :
1) As you mentionned yourself the approach I used in Damy is different as most of the other programmers. Let's take a very simple example. In the 5WK-2BK db almost all positions are winning positions for white. As a consequence the corresponding db in Damy is made of two files : the files of draw positions and the file of lost positions for white (this last file is empty if is white to move!).
As you can see I do not use a db based on 2 bits per position.
2) In addition why 4 states (Win/Draw/Lose/Unknown) instead of 3(Win/Draw/Lose) ?
3) For now several years, all world championship (and it will also be the case for the next wch in Brazil) use for tie break criteria the notion of draw+ and draw-.
I think that the rule for computers should be quite identical to the rule used in the major human tournaments. In that case we have to handle 5 states (Win,Draw+,Draw,Draw-,Lose).
I generated in the past the corresponding 5pieces db and I intend now to generate the 6p and 7p db.
4) Is compression so important ? For storing the 7p db with 5 states per position, without compression, we need about 450Gb. It was too high some years ago but it is not so high now. On my new computer I have a disk of 1,5Tb. For the 8p pieces db we need 8,5Tb. It is a little to high today but certainly not in 2 years!
Gérard
- Should things like 'should the 25-move rule be used in DB generation' be included in this standard?
- AFAIK, for long wins, knowing WDL may not be enough to actually win, because you don't know which move will actually bring you closer to the real win. So DTW/DTC databases are necessary too.
PS: The original post has 'dash dash >' in the text, which seriously mucks up the layout of the page, because the renderer thinks it's a comment end...
- AFAIK, for long wins, knowing WDL may not be enough to actually win, because you don't know which move will actually bring you closer to the real win. So DTW/DTC databases are necessary too.
PS: The original post has 'dash dash >' in the text, which seriously mucks up the layout of the page, because the renderer thinks it's a comment end...
Last edited by ildjarn on Tue Sep 29, 2009 10:36, edited 1 time in total.
Lasst die Maschinen verhungern, Ihr Narren...
Lasst sie verrecken!
Schlagt sie tot -- die Maschinen!
Lasst sie verrecken!
Schlagt sie tot -- die Maschinen!
-
Ed Gilbert
- Posts: 859
- Joined: Sat Apr 28, 2007 14:53
- Real name: Ed Gilbert
- Location: Morristown, NJ USA
- Contact:
Hi Bert,Some time ago Jaap Bus asked me if it was possible to standardize on EndGame databases.
Jaap uses multiple computer programs , to analyze (for example) endgame compositions.
To name a few (and most likely this list is not complete):
- Truus, Damage, KingsRow, TurboDambase, Dam 2.x, Flits.
All these programs using different EndGame Databases which can not be exchanged between Draughts programs.
At the moment this question is rather moot, because the only programs that have large databases are not publicly available. There is only a very small number of people that I have given copies of kingsrow to, and I assume the situation is the same with damy and damage. This situation will work itself out when one or more of the owners of a large db decides to 1) distribute the databases, and 2) document the internal organization, indexing, compression, etc. of the db, or distribute an access library for it.
For 8x8 checkers, you can download a dll from my web site which gives an API to several large databases, including the kingsrow 10pc db for English checkers, the 10pc db for Italian checkers, the Martin Fierz 8pc db, and the Chinook 8pc db. If I ever decide to distribute the 8/9pc draughts db I would probably do something similar.
-- Ed
